# Backends

One section per backend: its extra, tier and flags, the locator fields it reads, its `[backends.<kind>]` config, its state, how pins and forks map onto the system, its handle, and what `diff --content` reports. Credentials are never stored in manifests.

| Kind | Extra | Maturity | Tier | Flags |
|----|----|----|----|----|
| [`file`](#file-local-paths-and-object-stores) | - (local) / `objectstore` | stable | Observed; Addressable for a versioned remote object | [CHEAP_FINGERPRINT](../reference/testing.Capability.md#tether.testing.Capability.CHEAP_FINGERPRINT), [DIFF](../reference/testing.Capability.md#tether.testing.Capability.DIFF), [CREATE](../reference/testing.Capability.md#tether.testing.Capability.CREATE) (local directory) |
| [`icechunk`](#icechunk-icechunk-repositories) | `icechunk` | stable | Forkable | [ATOMIC_REF](../reference/testing.Capability.md#tether.testing.Capability.ATOMIC_REF), [DIFF](../reference/testing.Capability.md#tether.testing.Capability.DIFF), [HISTORY](../reference/testing.Capability.md#tether.testing.Capability.HISTORY), [PROMOTE](../reference/testing.Capability.md#tether.testing.Capability.PROMOTE), [CREATE](../reference/testing.Capability.md#tether.testing.Capability.CREATE); [CONDITIONAL_REF](../reference/testing.Capability.md#tether.testing.Capability.CONDITIONAL_REF) on `s3://` |
| [`git`](#git-git-and-jj-repositories) | - | stable | Forkable | [CHEAP_FINGERPRINT](../reference/testing.Capability.md#tether.testing.Capability.CHEAP_FINGERPRINT), [ATOMIC_REF](../reference/testing.Capability.md#tether.testing.Capability.ATOMIC_REF), [CONDITIONAL_REF](../reference/testing.Capability.md#tether.testing.Capability.CONDITIONAL_REF), [DIFF](../reference/testing.Capability.md#tether.testing.Capability.DIFF), [HISTORY](../reference/testing.Capability.md#tether.testing.Capability.HISTORY), [PROMOTE](../reference/testing.Capability.md#tether.testing.Capability.PROMOTE), [MERGE](../reference/testing.Capability.md#tether.testing.Capability.MERGE), [CREATE](../reference/testing.Capability.md#tether.testing.Capability.CREATE) |
| [`lance`](#lance-lance-datasets) | `lance` | stable | Forkable | [ATOMIC_REF](../reference/testing.Capability.md#tether.testing.Capability.ATOMIC_REF), [DIFF](../reference/testing.Capability.md#tether.testing.Capability.DIFF), [HISTORY](../reference/testing.Capability.md#tether.testing.Capability.HISTORY) |
| [`delta`](#delta-delta-lake-tables) | `delta` | stable | Addressable | [CHEAP_FINGERPRINT](../reference/testing.Capability.md#tether.testing.Capability.CHEAP_FINGERPRINT), [RETENTION_BOUND](../reference/testing.Capability.md#tether.testing.Capability.RETENTION_BOUND), [DIFF](../reference/testing.Capability.md#tether.testing.Capability.DIFF), [HISTORY](../reference/testing.Capability.md#tether.testing.Capability.HISTORY) |
| [`neon`](#neon-neon-postgres) | `neon` | experimental | Forkable | [NEEDS_QUIESCENCE](../reference/testing.Capability.md#tether.testing.Capability.NEEDS_QUIESCENCE), [RETENTION_BOUND](../reference/testing.Capability.md#tether.testing.Capability.RETENTION_BOUND), [BRANCH_IS_STORAGE](../reference/testing.Capability.md#tether.testing.Capability.BRANCH_IS_STORAGE) |
| [`iceberg`](#iceberg-apache-iceberg-tables) | `iceberg` | experimental | Forkable | [RETENTION_BOUND](../reference/testing.Capability.md#tether.testing.Capability.RETENTION_BOUND), [DIFF](../reference/testing.Capability.md#tether.testing.Capability.DIFF), [HISTORY](../reference/testing.Capability.md#tether.testing.Capability.HISTORY), [PROMOTE](../reference/testing.Capability.md#tether.testing.Capability.PROMOTE) |
| [`ducklake`](#ducklake-ducklake-catalogs) | `ducklake` | experimental | Addressable | [CHEAP_FINGERPRINT](../reference/testing.Capability.md#tether.testing.Capability.CHEAP_FINGERPRINT), [RETENTION_BOUND](../reference/testing.Capability.md#tether.testing.Capability.RETENTION_BOUND), [DIFF](../reference/testing.Capability.md#tether.testing.Capability.DIFF), [HISTORY](../reference/testing.Capability.md#tether.testing.Capability.HISTORY) |
| [`dolt`](#dolt-dolt-databases) | `dolt` | experimental | Forkable | [ATOMIC_REF](../reference/testing.Capability.md#tether.testing.Capability.ATOMIC_REF), [DIFF](../reference/testing.Capability.md#tether.testing.Capability.DIFF), [HISTORY](../reference/testing.Capability.md#tether.testing.Capability.HISTORY), [PROMOTE](../reference/testing.Capability.md#tether.testing.Capability.PROMOTE), [MERGE](../reference/testing.Capability.md#tether.testing.Capability.MERGE) |
| [`memory`](#memory-in-process-reference-backend) | - | stable | Forkable | [CHEAP_FINGERPRINT](../reference/testing.Capability.md#tether.testing.Capability.CHEAP_FINGERPRINT), [ATOMIC_REF](../reference/testing.Capability.md#tether.testing.Capability.ATOMIC_REF), [CONDITIONAL_REF](../reference/testing.Capability.md#tether.testing.Capability.CONDITIONAL_REF), [DIFF](../reference/testing.Capability.md#tether.testing.Capability.DIFF), [HISTORY](../reference/testing.Capability.md#tether.testing.Capability.HISTORY), [PROMOTE](../reference/testing.Capability.md#tether.testing.Capability.PROMOTE), [MERGE](../reference/testing.Capability.md#tether.testing.Capability.MERGE), [CREATE](../reference/testing.Capability.md#tether.testing.Capability.CREATE) (tests only) |

*Maturity* is what the backend declares ([MATURITY](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.MATURITY)), and `tether backends` prints it. **Stable** means the full lifecycle (register, commit, fork, write, commit, promote, gc) runs in CI against the real system: embedded stores and libraries, on local storage. The S3, GCS and Azure paths of `file` and `icechunk` are **not yet cloud-tested**; `ROADMAP.md` tracks that. **Experimental** means tested against a fake of a network service, not the service itself, and [add](../reference/testing.ObjectDiff.md#tether.testing.ObjectDiff.add) says so. The tiers and flags are defined in [Writing a Backend](extending.md#capabilities-decide-behavior).


# Conventions

- [branch](../reference/handles.NeonHandle.md#tether.handles.NeonHandle.branch) in a locator is the *upstream* branch (default `main`), which the trunk bookmark stands for. Other bookmarks fork `tether.ws.<dataset8>.<bookmark>` from the pin.
- Pins are named `tether.<pin_id>`, where `pin_id` is `<dataset8>.<hash16>`: the dataset's id, then a content address of kind, identity and state. Backends treat the id as opaque.
- [identity](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.identity) is the subset of the locator that names the system. Two objects of one dataset with the same identity and state share one pin, which [commit](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.commit) cuts once; the second object is recorded at the state the pin was taken at.
- A local path's identity is its resolved form: symlinks followed, `.`, `..`, doubled and trailing slashes gone, and `file://` folded. So `/p`, `/p/`, `file:///p` and a path through a symlinked parent are one store to pin ids, listings and `gc`. Re-pointing a symlink therefore changes the store's identity: its pins stay safe (`gc` never releases a pin any manifest names), but listings stored under the old identity are not found.
- Locators are committed and read from every directory and clone, so a *relative* local path is resolved against the directory [add](../reference/testing.ObjectDiff.md#tether.testing.ObjectDiff.add) or `import` ran in, and stored absolute; a leading `~` is expanded. That covers `file` [uri](../reference/handles.FileHandle.md#tether.handles.FileHandle.uri)/[path](../reference/handles.GitHandle.md#tether.handles.GitHandle.path), `git` [path](../reference/handles.GitHandle.md#tether.handles.GitHandle.path), the `icechunk`, `lance` and `delta` [uri](../reference/handles.FileHandle.md#tether.handles.FileHandle.uri), and DuckLake's [data_path](../reference/handles.DuckLakeHandle.md#tether.handles.DuckLakeHandle.data_path) and the path in [metadata](../reference/handles.DuckLakeHandle.md#tether.handles.DuckLakeHandle.metadata) (after `ducklake:`, `duckdb:` or `sqlite:`). URLs and absolute paths are stored as given.
- Objects in one *branch scope* (two databases of a Neon project, two keys on one Icechunk store) share a branch per bookmark, and `gc` collects their references per scope.
- `at` is an object's initial position (`tether add --at` / `--pick`), pinned by the first commit until a trunk `pull` moves the object onto the branch. What each system accepts, and what `tether log` lists:

| Kind | `at` accepts | `log` lists |
|----|----|----|
| `icechunk` | snapshot id, tag, branch | `Repository.ancestry` from the ref: id, `written_at`, message |
| `lance` | version number (on the base branch) or tag | `versions()` of the branch, newest first, tags on each |
| `iceberg` | snapshot id or ref name | snapshot ancestry with operation and record counts |
| `delta` | version number | transaction log with operation and metrics |
| `ducklake` | snapshot id | `ducklake_snapshots` with each snapshot's changes |
| `dolt` | commit hash, tag, or branch (`HASHOF`) | `DOLT_LOG(ref)` |
| `git` | any commit-ish (alias of [ref](../reference/handles.DoltHandle.md#tether.handles.DoltHandle.ref)) | `git log` with decorations |
| `neon`, `file` | not supported | - |


# What the flags mean in practice

**[PROMOTE](../reference/testing.Capability.md#tether.testing.Capability.PROMOTE) and [MERGE](../reference/testing.Capability.md#tether.testing.Capability.MERGE).** [PROMOTE](../reference/testing.Capability.md#tether.testing.Capability.PROMOTE) fast-forwards the base branch to a fork (Icechunk `reset_branch`, Iceberg `set-snapshot-ref`, git `update-ref` or `merge --ff-only`, Dolt merges). [MERGE](../reference/testing.Capability.md#tether.testing.Capability.MERGE) three-way merges a diverged fork (git [merge](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.merge), `DOLT_MERGE`). Lance cannot move a branch head yet, and Neon cannot promote a child branch, so both are refused with instructions. Where Icechunk and Lance stand upstream on branch merge, and what blocks it, is a dated survey in [`docs/upstream-branch-merge.md`](https://github.com/Rosebud-Biosciences/tether/blob/main/docs/upstream-branch-merge.md).

**[CONDITIONAL_REF](../reference/testing.Capability.md#tether.testing.Capability.CONDITIONAL_REF)** means a fork, promote or merge moves the ref only from the head the plan reviewed, in one atomic step. The others compare just before the move ([conditional moves](cli.md#conditional-moves)).

**[CREATE](../reference/testing.Capability.md#tether.testing.Capability.CREATE)** *(experimental)* is what `tether add --create` needs ([Stores tether created](reclaiming-storage.md#stores-tether-created)). Lance, Delta and Iceberg tables need a schema, so they cannot be created empty. Neon, Dolt and DuckLake objects live inside a project, database or catalog tether does not own. Make those with their own tools and [add](../reference/testing.ObjectDiff.md#tether.testing.ObjectDiff.add) them. `--pin record` removes [PIN](../reference/testing.Capability.md#tether.testing.Capability.PIN) from any backend ([Lever 1](reclaiming-storage.md#lever-1-do-not-pin-what-you-will-not-need---pin-record)).


# State keys: content and address

A state's keys do two jobs:

- **Content** keys say what the data is: an Icechunk snapshot id, a Neon `commit_xid`, a git sha. Equality, pin ids and listing names use only these.
- **Address** keys say where to find it, and can move without the data changing: Neon's `lsn`, and git's [change_id](../reference/vcs.CommitInfo.md#tether.vcs.CommitInfo.change_id) and [dirty](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.dirty). Backends list them in [VOLATILE_KEYS](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.VOLATILE_KEYS); [open](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.open), [pin](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.pin) and [verify](../reference/ObjectStatus.md#tether.ObjectStatus.verify) get the whole state.


# Content diffs

`tether diff A B` compares manifests. `--content` also asks each changed object's backend for its *native* diff, from metadata only, concurrently:

| Kind | Unit | Source | Example detail |
|----|----|----|----|
| `file` (dir / prefix) | files | two stored listings (`.tether/listings/`) | `modified a.tif +4.0 KiB` |
| `file` (object) | objects | recorded size / etag / version | `etag a -> b, version v1 -> v2` |
| `git` | files | `git diff --name-status` + `--numstat` | `modified main.py +1 -1` |
| `icechunk` | nodes | `Repository.diff(from, to)` | `modified /x 2 chunks` |
| `iceberg` | snapshots | snapshot summaries along `b`'s ancestry to `a` | `append: +5 rows, +1 files` |
| `delta` | commits | transaction-log [history()](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.history) in `(a, b]` | `WRITE: +2 rows, +1 files` |
| `lance` | fragments | fragment ids / deletions / schema at both versions | `added fragment 1 +1 rows` |
| `ducklake` | tables | snapshot `changes` + `ducklake_table_changes` | `modified main.t +5000 rows, -1 rows` |
| `dolt` | tables | `dolt_diff_summary` + `dolt_diff_stat` | `modified t +6 rows` |
| `neon` | - | no native diff |  |

A directory or prefix state is only a digest, so at commit time the `file` backend also hands the engine a per-file listing. It is stored content-addressed under `.tether/listings/` and committed with the manifests, like DVC's `.dir` files. `diff --content` compares two listings file by file, or falls back to a count and size summary. Entries are capped at 2000 per object; the counts are exact.


# `file`: local paths and object stores

Plain artifacts: a local file or directory, or an object / prefix in S3 (`s3://`), GCS (`gs://`), or Azure Blob (`az://`, `abfs://`), through [obstore](https://developmentseed.org/obstore/).

|  |  |
|----|----|
| Locator | [uri](../reference/handles.FileHandle.md#tether.handles.FileHandle.uri) (positional; [path](../reference/handles.GitHandle.md#tether.handles.GitHandle.path) accepted). A trailing `/` (or empty key) means a prefix. `region` overrides the store region. |
| Config | `storage_options = {...}` passed to `obstore.store.from_url`. Committed: only `region`, `allow_http`, `virtual_hosted_style_request` and `conditional_put`; endpoints, account names and credentials only in `.tether/secrets.toml`. HTTP client options (`allow_http`, timeouts, proxies, …) work in any spelling there (`AWS_ALLOW_HTTP`, `aws_allow_http`, or inside `client_options`); a number is a duration in seconds, and an unknown key raises [ConfigError](../reference/ConfigError.md#tether.ConfigError). Otherwise credentials come from the environment or instance metadata. |
| Identity | `{uri}`, a local path in its resolved form ([Conventions](#conventions)) |
| State | local file: `type=file, size, sha256`; local dir: `type=dir, count, size, digest` (digest over per-file sha256s); remote object: `type=object, size, etag[, version_id]`; prefix: `type=prefix, count, size, digest` (one paged `LIST`, etag+size per key). Local hashes are cached by `(size, mtime_ns, ctime_ns, inode)` in `.tether/cache/file-hashes.json`, so only files whose stat changed are re-read |
| Tier | Observed by default. `--file versioned` on a single *remote* object makes it Addressable via [version_id](../reference/handles.FileHandle.md#tether.handles.FileHandle.version_id) (S3 version id / GCS generation / Azure version id; requires versioning on the bucket/container - a recorded state without a version id is refused by [open](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.open)). Local paths and prefixes are always Observed. |
| Pin / fork | none |
| Handle | `FileHandle(uri, version_id)` - always read-only |
| Listing | directories and prefixes hand the engine a per-file JSONL listing (`{"p": path, "k": token, "s": size}`) that is stored under `.tether/listings/` |
| Diff | [files](../reference/vcs.GitObjectReader.md#tether.vcs.GitObjectReader.files): per-path added/removed/modified with size deltas from two stored listings; falls back to a count/size summary when a listing is missing; single objects report changed size/etag/sha256/version |
| Create | a local directory only (`mkdir` plus a `.tether-owner` marker file the directory walk skips, so the directory fingerprints as empty until something is written). Ref-empty means nothing but the marker; [delete_store](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.delete_store) is an `rmdir` that refuses anything else. Remote prefixes are not created or removed by tether. |
| Notes | `--file immutable` (default) turns any drift into an [ImmutableObjectModified](../reference/ImmutableObjectModified.md#tether.ImmutableObjectModified) error at [status](../reference/backends.VerifyReport.md#tether.backends.VerifyReport.status); drift means the *bytes* changed, not the timestamp. A symlink to a file is read through. A symlink to a directory, or a dangling one, is recorded by its target: retargeting it is a change, but edits under a linked directory are not seen. Fingerprint cost: [Performance](caveats-and-performance.md#fingerprints-are-metadata-only-except-the-first-local-one). |


# `icechunk`: Icechunk repositories

|  |  |
|----|----|
| Locator | [uri](../reference/handles.FileHandle.md#tether.handles.FileHandle.uri) (local path or `s3://bucket/prefix`), [branch](../reference/handles.NeonHandle.md#tether.handles.NeonHandle.branch) (default `main`), `region` (S3) |
| Config | none committed. `.tether/secrets.toml` may set `profile`, `role_arn`, literal keys, `endpoint_url` and `region` per URI prefix or object ([two identities](configuration.md#advanced-two-stores-two-identities)); otherwise S3 credentials come from the environment (`from_env`). A profile or role becomes a `get_credentials` callback that refreshes the keys, so a long-lived session does not expire mid-write; an icechunk without the callback gets static keys, rebuilt when they rotate. For an S3-compatible server (SeaweedFS, MinIO), `allow_http = true` and `force_path_style = true` in the same entry reach every Icechunk and obstore call, `add --create` and [delete_store](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.delete_store) included. |
| Identity | `{uri}`, a local path in its resolved form |
| State | [snapshot_id](../reference/handles.IcebergHandle.md#tether.handles.IcebergHandle.snapshot_id) of the branch head |
| Pin | `create_tag("tether.<id>", snapshot_id)`; tags are immutable, and the default `expire_snapshots` keeps a tagged snapshot. `expire_snapshots(delete_expired_tags=True)` deletes the tag and the snapshot with it; [verify](../reference/ObjectStatus.md#tether.ObjectStatus.verify) then reports the pin `missing` |
| Fork | `create_branch(name, snapshot_id)` off the tag, or `reset_branch` if it exists. A conditional move uses `reset_branch(from_snapshot_id=)` (icechunk 2.x), which is atomic on object stores only: on local filesystem storage racing processes can each win |
| Handle | `IcechunkHandle(repository, session, branch, tag, snapshot_id)`; [session](../reference/handles.IcechunkHandle.md#tether.handles.IcechunkHandle.session) is writable for a branch, read-only for a tag/snapshot |
| Diff | `nodes`: `Repository.diff(from, to)` - new/deleted groups and arrays, updated metadata, updated chunk counts per array, moves |
| Create | `Repository.create`, refused if anything exists at the path or under the prefix. The owner marker is the `tether.owner` key in the repository's metadata; an icechunk without repository metadata (2.2.0 works) lacks [CREATE](../reference/testing.Capability.md#tether.testing.Capability.CREATE). Ref-empty: no branch but the base, no tag beyond the pins the plan names, and the base still at the first snapshot. [delete_store](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.delete_store) re-checks owner and emptiness, then removes only Icechunk's own top-level names; anything else there is refused and kept. |


# `neon`: Neon Postgres

*Experimental*: tested against a fake of the Neon API, not a live project.

|  |  |
|----|----|
| Locator | `project_id`, [database](../reference/handles.DoltHandle.md#tether.handles.DoltHandle.database), `role` (all required, since Neon builds no connection URI without them), [branch](../reference/handles.NeonHandle.md#tether.handles.NeonHandle.branch) (default `main`) |
| Config | `api_key_env` (default `NEON_API_KEY`), `api_url` (default Neon API v2), `protected_pins` (default `false`; see [Caveats: Neon](caveats-and-performance.md#neon)), in `.tether/secrets.toml` only |
| Identity | `{project_id, branch}`: a Neon branch at an LSN is a snapshot of the whole project, so [database](../reference/handles.DoltHandle.md#tether.handles.DoltHandle.database) and `role` do not change pin ids. Two objects on two databases of one project share one pin per commit, and one working branch per bookmark |
| State | `lsn`, `commit_xid`, [branch](../reference/handles.NeonHandle.md#tether.handles.NeonHandle.branch), `timeline`, probed over SQL. Content is `commit_xid` (the newest *committed* transaction id) and [branch](../reference/handles.NeonHandle.md#tether.handles.NeonHandle.branch) (the lineage); `lsn` and `timeline` (the branch the LSN was read on) are volatile. With no commit among the last 65536 ids, `commit_xid` reads `<N`, so a later commit still shows as a change. What this means for equality: [Caveats: Neon](caveats-and-performance.md#neon) |
| Pin | a child branch `tether.<id>` of the state's timeline at its LSN (so a fork's pin hangs off the fork, not `main`), protected only with `protected_pins = true`; [RETENTION_BOUND](../reference/testing.Capability.md#tether.testing.Capability.RETENTION_BOUND) by the project's history retention |
| Fork | a child branch of the pin. Re-forking an existing working branch restores it onto the source. Once a pin hangs off the working branch Neon will not restore it in place, so the fork lands on a sibling (`<name>.2`, `.3`, …) while the old branch stays the pins' parent, and the engine records the returned name. A branch tether reads or writes gets one `read_write` endpoint, created once however many objects ask at the same time; pins and time-travel reads get a [read_only](../reference/testing.Handle.md#tether.testing.Handle.read_only) one, and each connection URI names its endpoint |
| Quiescence | [commit](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.commit) refuses while `pg_stat_activity` shows active writers (`--force` skips) |
| Handle | `NeonHandle(url, branch)` - a Postgres connection URL |
| Diff | not supported (no native primitive) |
| Notes | pins are branches, and branches are the storage: what that means for `gc`, [new](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.new), [promote](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.promote), cost and fingerprinting is in [Caveats: Neon](caveats-and-performance.md#neon). |


# `git`: git and jj repositories

Pin the *code* (or any git-tracked data) that produced a dataset. Colocated jj repositories work through their `.git`.

|  |  |
|----|----|
| Locator | [path](../reference/handles.GitHandle.md#tether.handles.GitHandle.path) (local checkout, stored absolute, and outside the dataset's own checkout; the positional locator works too: `tether add code --kind git ../code`), [ref](../reference/handles.DoltHandle.md#tether.handles.DoltHandle.ref) (default `HEAD`), `remote` (push pins there: the name of a remote the repository has configured; a URL or path only as `[objects."<key>"] remote` in `.tether/secrets.toml`), `allow_dirty` (pin despite uncommitted changes) |
| Config | `git_path`, `jj_path` - `.tether/secrets.toml` `[vcs]` or `TETHER_GIT`/`TETHER_JJ` only; a committed value is refused |
| Identity | `{path}` (resolved absolute) |
| State | [sha](../reference/handles.GitHandle.md#tether.handles.GitHandle.sha); [dirty](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.dirty) (only meaningful - and only computed - for the checked-out ref); [change_id](../reference/vcs.CommitInfo.md#tether.vcs.CommitInfo.change_id) when jj is available. Both [dirty](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.dirty) and [change_id](../reference/vcs.CommitInfo.md#tether.vcs.CommitInfo.change_id) are volatile: the same sha is the same state - and pins identically - whether or not jj is installed or the checkout has stray files; [pin](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.pin) still reads the full state and refuses a dirty tree |
| Pin | lightweight tag `tether.<id>` at the sha. With `remote` set the pin is pushed and is only a pin once the push succeeds: a failed push fails the [pin](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.pin) and removes the local tag; [unpin](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.unpin) deletes on the remote first and fails if that fails, so local and remote never diverge silently |
| Fork | branch `name` at the tag's sha. A conditional move is `update-ref` with the old value, atomic; a branch checked out in any worktree is refused |
| Handle | `GitHandle(path, sha, worktree)` |
| Diff | [files](../reference/vcs.GitObjectReader.md#tether.vcs.GitObjectReader.files): `git diff --name-status -M` with `--numstat` line counts (`+a -d`) |
| Create | `git init -b <ref>` plus one empty root commit with a fixed tree, author and dates, so every created store has the same root sha (`GitBackend.EMPTY_ROOT_SHA`); the owner marker is `git config tether.owner`. Ref-empty: no branch or tag beyond the base, the base *at* that root sha, and a clean working tree, since an untracked, staged or ignored file is data. [delete_store](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.delete_store) re-checks both before it removes the directory. |
| Notes | remote-only (url) repositories are not supported yet. Every git call runs with `core.fsmonitor`, hooks and `ext::` transports off and `safe.bareRepository=explicit`, so a repository's config runs nothing; git 2.38 is required. [promote](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.promote) and [merge](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.merge) move a branch. With the default `ref = HEAD` that is the checked-out branch, and a detached `HEAD` (a colocated jj checkout always has one) is refused: set [ref](../reference/handles.DoltHandle.md#tether.handles.DoltHandle.ref) to the branch name. A merge, or a promote onto a checked-out base, compares the head and then merges in the checkout, so a commit landing in between is merged or refused, never discarded. |


# `iceberg`: Apache Iceberg tables

*Experimental*: driven through a fake of the pyiceberg table API; not yet run against a catalog service.

|  |  |
|----|----|
| Locator | `identifier` (`ns.table`; [table](../reference/handles.DeltaHandle.md#tether.handles.DeltaHandle.table) accepted), [branch](../reference/handles.NeonHandle.md#tether.handles.NeonHandle.branch) (default `main`), `catalog_name` (default `default`), `catalog` (dict of pyiceberg catalog properties, or set once in config) |
| Config | `catalog = {...}` pyiceberg properties. Committed (`tether.toml` or the locator): exactly `type`, `warehouse` and `name`; every other property - [uri](../reference/handles.FileHandle.md#tether.handles.FileHandle.uri), `token`, `credential`, … - only in `.tether/secrets.toml` (`[backends.iceberg] catalog` or per identifier) |
| Identity | `{identifier, branch, catalog_name}` |
| State | [snapshot_id](../reference/handles.IcebergHandle.md#tether.handles.IcebergHandle.snapshot_id) (nothing else: `metadata_location` changes on every table commit on any branch). A new table with no snapshot yet has `snapshot_id = -1`, Iceberg's "no current snapshot" |
| Pin | tag `tether.<id>`; with `--pin record`, the snapshot id alone, recoverable within snapshot retention (for S3 Tables and other catalogs where user refs disable maintenance). The empty state needs no tag: its pin verifies as held and opens at `snapshot_id = -1`, which pyiceberg's scans refuse rather than read today's data |
| Fork | `create_branch(name, snapshot_id)`; an existing name is re-pointed (`set-snapshot-ref`). A table with no snapshot cannot be forked, so write it on the trunk first. Needs pyiceberg 0.11 or newer |
| Handle | `IcebergHandle(table, ref, snapshot_id)` |
| Diff | `snapshots`: one entry per snapshot on `b`'s ancestry back to `a`, from snapshot summaries (`append: +5 rows, +1 files`); when `a` is not an ancestor, `total-*` counters are compared instead |


# `delta`: Delta Lake tables

|  |  |
|----|----|
| Locator | [uri](../reference/handles.FileHandle.md#tether.handles.FileHandle.uri) (positional), `region` (sets `AWS_REGION`) |
| Config | `storage_options = {...}` passed to `DeltaTable`; committed, the same four keys as `file`, and the rest in `.tether/secrets.toml` |
| Identity | `{uri}`, a local path in its resolved form |
| State | [version](../reference/handles.DeltaHandle.md#tether.handles.DeltaHandle.version), `table_id` (metadata id - a recreated table verifies as drifted) |
| Pin / fork | none: Delta has no tags or branches. Versions stay readable until `VACUUM` (7-day default) / log retention (30 days). |
| Handle | `DeltaHandle(uri, version, table)` - read-only `DeltaTable` at the version |
| Diff | `commits`: one entry per transaction-log commit in `(a, b]` with its operation and metrics (`WRITE: +2 rows, +1 files`) |


# `lance`: Lance datasets

|  |  |
|----|----|
| Locator | [uri](../reference/handles.FileHandle.md#tether.handles.FileHandle.uri) (positional), [branch](../reference/handles.NeonHandle.md#tether.handles.NeonHandle.branch) (default `main`) |
| Config | `storage_options = {...}` passed to `lance.dataset`; committed, the same four keys as `file`, and the rest in `.tether/secrets.toml` |
| Identity | `{uri}`, a local path in its resolved form |
| State | [branch](../reference/handles.NeonHandle.md#tether.handles.NeonHandle.branch), [version](../reference/handles.DeltaHandle.md#tether.handles.DeltaHandle.version), and on any branch but `main` its `branch_id` (Lance version numbers are branch-scoped and restart when a branch is deleted and re-created, so the id tells two lives of one name apart; `--at N` means version `N` of the locator's [branch](../reference/handles.NeonHandle.md#tether.handles.NeonHandle.branch) - it is a *base* coordinate and is never consulted once a working branch exists; use a tag to name a version on another branch) |
| Pin | `tags.create("tether.<id>", (branch, version))`; tagged versions are exempt from `cleanup_old_versions()` |
| Fork | `create_branch(name, tag)`. Lance refuses to delete a branch a tag references, so a pinned working branch outlives its workspace and a re-fork under the same name picks `name.2`, `name.3`, … A conditional fork onto an existing branch compares its head, then deletes and recreates it, which leaves a small window; one expecting "absent" is atomic |
| Handle | `LanceHandle(uri, dataset, version, branch, tag)`; append with `lance.write_dataset(table, handle.dataset, mode="append")` |
| Diff | `fragments`: fragment ids added/removed, deletion-file changes, columns added/removed, from the two manifests |
| Notes | an untouched fork reports its parent's `(branch, version)`, so a no-op commit after `tether new` is a no-op. |


# `ducklake`: DuckLake catalogs

*Experimental*: tested against a local DuckDB-file catalog, not a Postgres or MySQL catalog with an object-store [data_path](../reference/handles.DuckLakeHandle.md#tether.handles.DuckLakeHandle.data_path).

|  |  |
|----|----|
| Locator | [metadata](../reference/handles.DuckLakeHandle.md#tether.handles.DuckLakeHandle.metadata) (positional [uri](../reference/handles.FileHandle.md#tether.handles.FileHandle.uri) accepted; the `ducklake:` prefix is added if missing, e.g. `ducklake:postgres:dbname=lake`), [data_path](../reference/handles.DuckLakeHandle.md#tether.handles.DuckLakeHandle.data_path), [table](../reference/handles.DeltaHandle.md#tether.handles.DeltaHandle.table) (scopes the handle only; snapshots are catalog-wide). A DuckDB or SQLite file path in [metadata](../reference/handles.DuckLakeHandle.md#tether.handles.DuckLakeHandle.metadata) (bare, `duckdb:` or `sqlite:`), and [data_path](../reference/handles.DuckLakeHandle.md#tether.handles.DuckLakeHandle.data_path), are stored absolute with `~` expanded; a pre-0.1.0b4 relative path is made absolute by `tether upgrade` |
| Config | `init_sql = ["CREATE SECRET ...", ...]` run on every fresh connection - `.tether/secrets.toml` only (it runs code); credentials there or in the environment, never in the locator |
| Identity | `{metadata[, data_path]}` |
| State | [snapshot_id](../reference/handles.IcebergHandle.md#tether.handles.IcebergHandle.snapshot_id), `snapshot_time_us` (epoch microseconds; a recreated catalog verifies as drifted) |
| Pin / fork | none: DuckLake has no tags or branches. `ducklake_expire_snapshots` bounds how long a snapshot stays readable. |
| Handle | `DuckLakeHandle(metadata, alias, connection, snapshot_id, attach_sql, data_path, table)`: a DuckDB connection with the catalog attached read-only (at `SNAPSHOT_VERSION` for a recorded state); [table_ref()](../reference/handles.DuckLakeHandle.md#tether.handles.DuckLakeHandle.table_ref) gives `alias.table`; `close()` / `with handle:` detaches |
| Diff | `tables`: per-table created/dropped/altered/inserted/deleted from the snapshots' `changes` maps, with row counts from `ducklake_table_changes` for tables that still exist. A table is named by its path, a part that is not a plain identifier double-quoted: `sales.europe.orders`, `"a.child".t` |
| Notes | a DuckDB-file metadata catalog can be attached only once per process: operations attach/detach per call (serialized per catalog), and an open handle or your own writer connection in the same process blocks [status](../reference/backends.VerifyReport.md#tether.backends.VerifyReport.status). Postgres/SQLite/MySQL catalogs have no such limit. |


# `dolt`: Dolt databases

*Experimental*: tested against a fake of the Dolt SQL surface, not a running `dolt sql-server`.

|  |  |
|----|----|
| Locator | `host` (default `127.0.0.1`), `port` (default 3306), [database](../reference/handles.DoltHandle.md#tether.handles.DoltHandle.database) (required), [branch](../reference/handles.NeonHandle.md#tether.handles.NeonHandle.branch) (default `main`), `user`; or `url = mysql://user@host:port/database` |
| Credentials | only the server's `[uris."mysql://host:port"]` entry in `.tether/secrets.toml` (a `mysql://host:port/database` entry beats it): `user` and `password`, or `user_env` and `password_env`. A manifest chooses the host, so a server with no entry gets no password (and user `root` unless the locator names one) |
| Identity | `{host, port, database}` |
| State | [commit](../reference/vcs.VcsAdapter.md#tether.vcs.VcsAdapter.commit) (branch head hash), plus `dirty = true` when `dolt_branches.dirty` reports uncommitted working-set changes (refused at pin time) |
| Pin | `CALL DOLT_TAG('tether.<id>', commit)` |
| Fork | `CALL DOLT_BRANCH(name, tag)` (recreated with `-D` if it exists at another commit) |
| Promote / merge | `DOLT_MERGE` (`--ff-only` to promote) inside an explicit transaction, the only way Dolt reports which tables conflict or violate a constraint. Those come back as [MergeConflict](../reference/MergeConflict.md#tether.MergeConflict), and the merge is rolled back |
| Handle | `DoltHandle(url, database, ref, commit)`: a revision-database URL `mysql://user@host:port/db/<branch|tag|hash>` (tags and hashes are read-only in Dolt); [database_ref](../reference/handles.DoltHandle.md#tether.handles.DoltHandle.database_ref) is `db/ref` for `USE` |
| Diff | `tables`: `dolt_diff_summary` (added/dropped/modified/renamed tables, schema changes) with `dolt_diff_stat` row counts |
| Notes | talks to `dolt sql-server` / Hosted Dolt via PyMySQL; the SQL surface sits behind the `DoltClient` protocol (`tether.experimental.backends.dolt`) so other transports can be substituted. |


# `memory`: in-process reference backend

The executable specification of the protocol, used by the conformance suite and the engine tests. Locator `{system, branch}`; state [snapshot_id](../reference/handles.IcebergHandle.md#tether.handles.IcebergHandle.snapshot_id); tags and branches in a process-global `MemoryStore` (`tether.backends.memory.default_store()`). [MemoryHandle.read()](../reference/handles.MemoryHandle.md#tether.handles.MemoryHandle.read) / `.write(payload)` operate on the current ref; [diff](../reference/testing.ObjectBackend.md#tether.testing.ObjectBackend.diff) reports changed payload keys.