Configuration

What a dataset is made of on disk, what tether.toml can say, where credentials and other per-checkout settings live, and what a cloned dataset can and cannot make tether do.

Layout

<dataset-root>/
  tether.toml                 # committed: format version, dataset id, snapshot/verify/new settings,
                              #   default policies, non-secret backend options
  .tether/
    .gitignore                # ignores workspace.toml, ops.jsonl, secrets.toml, cache/, lock
    secrets.toml              # per checkout: credentials or references, endpoints, executables, SQL
    objects/<key>.toml        # committed, one per object: kind, locator, policy, state, pin,
                              #   recoverable, captured_at, origin (only when "created")
    listings/<hash>.jsonl     # committed, content-addressed per-file listings of directory and
                              #   prefix states (for `diff --content`); pruned by gc
    workspace.toml            # per checkout: workspace id, bookmark, working refs, pending forks,
                              #   base states, fork points, last snapshot
    ops.jsonl                 # per checkout: the operation log

<vcs shared dir>/           # .jj/repo/ or .git/: one per clone, shared by its checkouts, never committed
  tether.lock                 # the repository-wide lock (see Caveats)
  tether-pinned.jsonl         # pins this clone created: the only ones gc releases by default
  tether-created.jsonl        # stores this clone made with `add --create`
  tether-touched.jsonl        # stores this clone forked a branch in or pinned

The three per-checkout files are refused while the VCS tracks them (the security model). The indexes live with the repository, not in history or a checkout: an abandoned fork takes the manifest that named a store with it, and forget-workspace takes the checkout’s files. They are memories, not authority. Reclaiming Storage says how gc uses them, and why a re-clone starts without them.

pin_id = <dataset-id>.<blake2b(kind, locator identity, content state)[:16]>, and the native ref is tether.<pin_id>. An identical state yields the same pin within a dataset: re-committing an unchanged object is a no-op, and identical states dedupe to one pin. The identity is the backend’s canonical form of the locator, such as a resolved local path (Backends).

`tether.toml`

A dataset’s configuration is tether.toml at the dataset root, committed with the manifests and loaded into RepoConfig. Every key but [dataset] id has a default. A dataset whose version is older than this tether’s is refused by every command except tether upgrade.

[tether]
version = 5         # the format this tether writes; older datasets need `tether upgrade`

[dataset]
id = "0a1b2c3d"      # 8 hex chars; the namespace of this dataset's pins and branches in every store

[snapshot]
auto = false         # `status` shows the last snapshot (--snapshot refreshes); true: fingerprint on every status

[verify]
on_status = false    # also run a cheap verify of each committed object in status

[new]
fork = "lazy"        # "lazy": create a working branch on first writable open; "eager": during `new`
record = "fork"      # pin=record objects: "fork" during `new` even when lazy; "defer": like the rest (needs fork = "lazy")

[defaults]           # policy for objects added without an explicit one
file = "immutable"   # immutable | versioned
pin = "native"       # native | record

[vcs]
prefer = "jj"        # which adapter to use when both .jj and .git exist
trunk = "main"       # the bookmark that stands for every object's upstream branch

[backends.file]      # non-secret options only; one table per kind
storage_options = { region = "us-west-2" }

Dataset id

init generates [dataset] id, and every clone shares it. Every native ref carries it (pins tether.<id>.<hash>, working branches tether.ws.<id>.<bookmark>), and gc stays inside that namespace. So datasets can share one Icechunk repository or Neon project without deleting each other’s pins. Datasets from before the id existed get one from tether upgrade.

tether init --dataset-id ID (Repo.init(dataset_id=...)) uses the id you give, which must be 8 lowercase hex characters. A dataset whose repository was lost gets its stores’ pins and branches back this way, and tether recover finds the id when you do not know it (Recovering a lost dataset repo).

Do not change the id after committing

Before anything is pinned the id can still change; that is how tether recover takes back a lost dataset’s refs. After that:

Pins and branches made under the old id stop being this dataset’s: gc treats them as foreign and leaves them alone.
repair recreates the pins the manifests name, under the new id.
A fork meant to become an independent dataset must change the id before it pins anything, or the two share a namespace and can gc each other.

Policies

A Policy is stored per object ([policy] in its manifest). It is set with the --file and --pin options of tether add (or Repo.add(policy=...)) and changed with tether set KEY... --file|--pin. Whether writes fork or land upstream is not a policy: the trunk writes upstream, and any other bookmark forks. Manifests from before 0.1.0a9 carry a write key, which is ignored and dropped by tether upgrade.

Field	Values	Meaning
`file`	`immutable` (default), `versioned`	`file` backend only. `immutable`: any drift is an error at status. `versioned`: a single object in a versioning-enabled bucket or container is Addressable through its version id.
pin	`native` (default), `record`	Any backend. `record` drops PIN: commit records the state without a native ref, and new forks from the recorded state. Cheaper, but recoverable only while the system keeps that state (Lever 1).

Backend options

[backends.<kind>] is handed to the backend when it is first used. The committed file may set only what a backend allowlists: storage_options for file, delta and lance, and catalog for iceberg. Inside those tables only the named keys pass (region, allow_http, virtual_hosted_style_request, conditional_put; type, warehouse, name). Any other key is refused by name, so an option tether has never heard of cannot reach a store from a clone. Everything else goes in .tether/secrets.toml, where the same tables may hold any key. A table set in both files is merged field by field, and secrets.toml wins a field both set. So a committed catalog keeps its type and warehouse when secrets.toml adds only the catalog’s uri. Per-backend keys: Backends.

Trust boundary: `.tether/secrets.toml`

A cloned dataset is untrusted input. tether.toml and the manifests arrive with every clone, so nothing in them may choose an executable, an endpoint your credentials go to, SQL to run, or the variable a secret is read from. tether refuses a committed tether.toml that tries ([vcs] git_path, [import] query, [backends.neon] api_url, [backends.ducklake] init_sql, a storage_options.endpoint, …), and says where the key belongs: .tether/secrets.toml, per checkout and never committed, or the environment.

[vcs]
git_path = "/usr/bin/git"                 # or TETHER_GIT / TETHER_JJ in the environment
# jj_path = "/opt/jj/bin/jj"

[backends.neon]                           # per-kind options the committed file may not set
api_key_env = "NEON_API_KEY"

[backends.ducklake]
init_sql = ["CREATE SECRET (TYPE S3, PROVIDER credential_chain)"]

[uris."s3://lab-a-bucket/"]               # every object whose store URI starts here
profile = "lab-a"                          # an AWS shared-config profile ...
# role_arn = "arn:aws:iam::123456789012:role/lab-a"   # ... or a role, assumed via STS ...
# access_key_id = "..."                    # ... or, when nothing else will do, literal keys
# secret_access_key = "..."
# endpoint_url = "http://seaweedfs.local:8333"
# allow_http = true                        # icechunk: plain HTTP to that endpoint
# force_path_style = true                  # icechunk: bucket in the path, not the host

[objects."zarr/imaging"]                  # one object; beats a URI prefix
profile = "imaging"

[uris."mysql://dolt.internal:3306"]       # a Dolt server: the only one sent this password
user = "svc"                              #   (Dolt reads server entries only)
password_env = "DOLT_PASSWORD"

[import]                                  # the default SQL for `tether import` (see Registries and SQL)
query = "SELECT name AS key, 'file' AS kind, uri FROM catalog"

Resolution per object and per field, the most specific first: [objects."<key>"], then the longest matching [uris."<prefix>"], then [backends.<kind>], then the environment and instance credential chain. A table given at two levels is merged, not replaced. Per-object entries also reach add --create and gc --delete-stores. file, delta and lance take the entries as storage options, and icechunk passes them to s3_storage. Dolt honours only the server’s [uris."mysql://host:port"] entry (or one for mysql://host:port/database), because the manifest chooses the host a password is sent to.

A profile or role_arn is resolved through boto3 (install it when you use them) into explicit credentials. Concurrent callers share one resolution, cached until five minutes before the keys expire. Icechunk takes a profile or role as a refreshing credential source, so a session held for hours gets fresh keys instead of failing mid-write.

The file may hold literal credentials, so tether warns when other users can read it (chmod 600) and never prints its contents: not in status, not in --json, not in an error. init lists it in .tether/.gitignore, and Repo.find adds the entry to an older dataset that lacks it.

Advanced: two stores, two identities

The case the per-URI and per-object entries exist for: a dataset tracks an Icechunk repository in the imaging lab’s account and another in the sequencing core’s, and one commit pins both. With from_env alone that is an AWS_PROFILE switch between commands, which one commit cannot do.

tether add zarr/imaging s3://imaging-lab-data/plate-01.icechunk --kind icechunk
tether add zarr/reads   s3://seqcore-archive/run-2026-09.icechunk --kind icechunk

# .tether/secrets.toml (this checkout only; never committed)
[uris."s3://imaging-lab-data/"]      # every object in the imaging bucket
profile = "imaging-lab"              # a profile in ~/.aws/config

[objects."zarr/reads"]               # one object, whichever bucket it is in
role_arn = "arn:aws:iam::210987654321:role/seqcore-reader"
region = "us-east-2"

tether commit now opens zarr/imaging with the imaging-lab profile’s keys and zarr/reads with a session assumed into the sequencing core’s role, in one process. A third object with no entry keeps the ambient environment, and the manifests record only the two URIs.

On CI the same layout works with literal keys the runner writes into the file before the first command. tether does not expand ${...} itself, so write the values in, or use a role the runner’s identity can assume (A CI recipe).

Security model

What a cloned dataset (its tether.toml, manifests, history, and every other file in the checkout) can and cannot make tether do.

It cannot:

Run a program. Executables come from secrets.toml or the environment. secrets.toml, workspace.toml and ops.jsonl are refused while the VCS tracks them, since a clone brings a tracked copy with it. find checks, and so does every later refresh (each open, each writing command) that finds workspace.toml changed. The check runs before secrets.toml is read, so a jj reachable only through its jj_path needs TETHER_JJ.
Run git code through a git object. The git backend runs with core.fsmonitor, hooks and ext:: transports off, and never takes a directory for a bare repository by itself (safe.bareRepository, which is why git 2.38 is required). A git object’s path must be absolute and outside the checkout, whose files came with the clone.
Choose where writes go, or which pins gc releases. Those are workspace.toml and the op log, refused when tracked. Another checkout’s tracked copy is skipped with a warning naming the checkout.
Send your credentials somewhere. Endpoints, and Icechunk’s allow_http and force_path_style, come from secrets.toml only. A Dolt password goes only to a server secrets.toml lists.
Run SQL. [import] query and init_sql come from secrets.toml or the command line. An import query runs as one statement in a read-only transaction.
Push your code. A git object’s remote must name a remote the repository already has.
Write outside .tether/. Object keys are checked when a manifest is read, not only at add: no .., no \, no drive letters.

It can:

Name stores. A locator is an address. A manifest can point status, commit and verify at any store your ambient credentials reach, and they sign the requests (a signature does not reveal the secret). commit records what it finds, a directory’s file names and hashes included, and pushing publishes that.
Release pins in your stores, with your help. gc releases only pins in its dataset’s namespace that no manifest names and that this clone created. --release-foreign drops the last condition. In a clone that reuses another dataset’s id and names its stores, that releases the other dataset’s pins.
Make commands fail. Every refusal above is an error: a hostile clone can stop a command, not redirect it.

So read a clone’s manifests before its first commit, and a gc plan before applying it. The dataset’s own repository runs your git hooks on dataset commits; those live in your .git/, which no clone brings.

Workspace state

.tether/workspace.toml is per checkout (per jj workspace or git worktree):

workspace_id = "7c1e0a4d9b2e4f6a8c1d3e5f7a9b0c2d"
bookmark = "relabel"                 # the dataset bookmark this checkout works on (absent: read-only)
last_snapshot_at = "2026-09-04T22:10:03.118Z"

[working_refs]                       # branches that exist (on the trunk: the upstream branches)
"zarr/imaging" = "tether.ws.0a1b2c3d.relabel"
"db/metrics" = "tether.ws.0a1b2c3d.relabel"

[pending_forks]                      # decided by `new`, created on first writable open
"features" = "tether.ws.0a1b2c3d.relabel"

[pending_resets."features"]          # the head `new` reviewed and agreed to reset on that open
version = 9                          #   (a branch that moved since is refused instead)

[base_states."zarr/imaging"]         # committed state this working ref corresponds to
snapshot_id = "K3F7Q2ZJ1XN4Y8W0"     #   (differs from the manifest => stale)

[fork_points."zarr/imaging"]         # base state when the branch was forked (for promote)
snapshot_id = "K3F7Q2ZJ1XN4Y8W0"

[last_snapshot."zarr/imaging"]       # what a plain `status` shows (errors are kept too)
snapshot_id = "K3F7Q2ZJ1XN4Y8W0"

A writable open moves an entry from pending_forks to working_refs. pending_resets keeps it from resetting a branch that moved since new, and base_states drive the stale check. Every open re-reads the file, so a long-lived Repo follows what another process’s new wrote.

Operation log

.tether/ops.jsonl is this checkout’s operation log: one JSON line per store-writing command, appended only, like jj’s op log. The VCS records the manifests, but it cannot see which refs commit created, which branches new forked or reset, what gc deleted, or where promote moved a base branch from. Each entry holds the plan it applied, what it did, and what it replaced. tether ops lists the entries, and tether undo reverses from them.

The log is a journal. A store-writing command appends its entry and syncs it before the first side effect, appends a progress record as each effect lands, and marks completion with the result. A process killed in between leaves an INCOMPLETE entry. undo refuses it, repair --dry-run says how far it got, and running the command again finishes what is left. The new and gc a drop runs are journaled as its steps (parent). An interruption loses at most the line being written, and the next entry starts on a fresh line.

Other commands read the log too, so deleting it loses more than undo:

gc keeps every pin an unfinished operation in any live checkout set out to create;
status and ops flag dataset commits that vanished from the VCS;
the first gc seeds the pin index from the logs, so pins whose entries are gone by then are kept as another clone’s.

Environment variables

Every variable tether reads (TETHER_REV, TETHER_JJ / TETHER_GIT, TETHER_NATIVE, TETHER_PUBLISH_DSN, NEON_API_KEY, the Dolt *_env names, the cloud credential chains) is in one table in the CLI Guide, with the GIT_* variables tether ignores.