Skills

A skill is a package of structured files that teaches an AI coding agent how to work with a specific tool or framework. The skill below was generated by Great Docs from this project’s documentation. Install it in your agent and it will be able to run commands, edit configuration, write content, and troubleshoot problems without step-by-step guidance from you.

Any agent — install with npx:

npx skills add https://rosebud-biosciences.github.io/tether/

Codex / OpenCode

Tell the agent:
Fetch the skill file at https://rosebud-biosciences.github.io/tether/skill.md and follow the instructions.

Manual — download the skill file:

curl -O https://rosebud-biosciences.github.io/tether/skill.md

Or browse the SKILL.md file.

SKILL.md

---
name: tether-vcs
description: >
  Version control (fingerprints, pins, and forks) for heterogeneous datasets: files and object stores, Icechunk, Lance, Delta, git, and -- experimental -- Neon, Iceberg, DuckLake, Dolt. Use when writing Python code that uses the tether package.
license: Apache-2.0
compatibility: Requires Python >=3.11.
---

# tether

Version control (fingerprints, pins, and forks) for heterogeneous datasets: files and object stores, Icechunk, Lance, Delta, git, and -- experimental -- Neon, Iceberg, DuckLake, Dolt

## Installation

```bash
pip install tether-vcs
```

## API overview

### Repository

The engine and the results it returns.

- `Repo`: A tether dataset: manifests in a VCS working tree plus the systems they name
- `StatusReport`: Result of `Repo.status`
- `ObjectStatus`: One object's classification in a `StatusReport`
- `VcsDrift`: A dataset commit that is gone from VCS history without tether knowing
- `CommitResult`: Result of `Repo.commit`
- `GcReport`: Result of `Repo.gc`
- `AbandonReport`: What `Repo.abandon` did
- `DropReport`: What `Repo.drop` did: a bookmark, its commits, and its store leftovers,
- `ForgetWorkspaceReport`: What `Repo.apply_forget_workspace` did
- `PullReport`: What `pull` did: the fetched heads of a bookmark's branches, committed
- `SetReport`: What `set` changed per object
- `PromoteReport`: Result of `Repo.apply_promote`
- `UndoReport`: What `Repo.undo` reversed, could not reverse, and left alone
- `RepairReport`: What `Repo.repair` rebuilt
- `RecoverReport`: What `Repo.recover_report` found: tether's refs in each object's store,
- `RecoverStep`: One step of a `RecoverReport`: a command to run, or none, and a note
- `RecoveredObject`: One object's row in a `RecoverReport`
- `RecoveredRefs`: The refs a store holds under one dataset id (see `RecoverReport`)
- `upgrade.UpgradeReport`
- `DiffEntry`: One object's row in `Repo.diff`
- `TETHER_REV_ENV`: str(object='') -> str

### Operation log

What tether did to the stores, per workspace; what undo works from.

- `OpEntry`: One applied operation
- `oplog.OpEntry`
- `oplog.TouchedStore`
- `oplog.tether_path`
- `oplog.new_op_id`
- `oplog.ops_path`
- `oplog.report_dict`
- `oplog.read_ops`
- `oplog.read_jsonl`
- `oplog.append_index_entry`
- `oplog.remove_index_entry`
- `oplog.touched_path`
- `oplog.append_touched`
- `oplog.read_touched`
- `oplog.remove_touched`
- `oplog.pinned_path`
- `oplog.read_pinned`
- `oplog.append_pinned`
- `oplog.remove_pinned`
- `oplog.mark_progress`
- `oplog.append_op`
- `oplog.mark_done`
- `oplog.mark_undone`
- `oplog.OPS_FILENAME`
- `oplog.TOUCHED_FILENAME`
- `oplog.PINNED_FILENAME`

### OpEntry Methods

Methods for the OpEntry class

- `OpEntry.now`
- `OpEntry.incomplete`
- `OpEntry.to_dict`
- `OpEntry.from_dict`
- `OpEntry.undoable`
- `OpEntry.summary`

### Migrations

The one composed migration behind `tether upgrade`, from any 0.1.0 alpha format to the current one. Ships with the betas and is removed at 0.1.0.

- `upgrade.Repo`
- `upgrade.Plan`
- `upgrade.Migration`
- `upgrade.UpgradeReport`
- `upgrade.TetherError`
- `upgrade.ensure_ignored`
- `upgrade.write_config`
- `upgrade.report_dict`
- `upgrade.pending`
- `upgrade.outdated_message`
- `upgrade.plan_upgrade`
- `upgrade.apply_upgrade`
- `upgrade.upgrade`
- `upgrade.CONFIG_VERSION`
- `upgrade.MIGRATIONS`
- `upgrade.LAST_BETA_WITH_UPGRADE`

### Plans

What a store-writing command would do, as data. `Repo.plan_*` returns a `Plan`; `Repo.apply_*` executes one after re-checking the world matches.

- `Plan`: The actions a command would perform, plus the inputs they depend on
- `Precondition`: One thing that must still hold when a plan is applied
- `Action`: One store-writing step in a `Plan`

### Plan Methods

Methods for the Plan class

- `Plan.writes`
- `Plan.is_empty`
- `Plan.require`
- `Plan.digest`
- `Plan.to_dict`
- `Plan.to_json`
- `Plan.from_dict`
- `Plan.edited`
- `Plan.from_json`
- `Plan.missing_preconditions`
- `Plan.render`

### Manifest model

What is committed to the VCS (`tether.toml`, `.tether/objects/*.toml`) and kept untracked (`.tether/workspace.toml`), plus the hashing and naming helpers behind pin ids and refs.

- `RepoConfig`: Committed repository configuration
- `Policy`: Per-object behavioral knobs
- `Pin`: A durable native reference created for a committed state
- `ObjectManifest`: The committed record for one object
- `WorkspaceState`: Untracked per-workspace working state
- `compute_pin_id`: Content-address a pin: ``<dataset8>.<hash16>``
- `ref_for_pin`: Native reference name for a pin id (dot-delimited; no ``/``)
- `working_ref_name`: The store branch that stands for a dataset bookmark
- `working_ref_bookmark`: The bookmark slug a working ref stands for; ``None`` for legacy
- `working_ref_workspace`: The 8-char workspace id in a *legacy* working ref name (before
- `listing_name`: Content-addressed file name for a state's listing
- `manifest_hash`: Hash the full committed object set (order-independent)

### Handles

Typed native handles returned by `Repo.open`.

- `handles.Handle`
- `handles.FileHandle`
- `handles.IcechunkHandle`
- `handles.NeonHandle`
- `handles.GitHandle`
- `handles.IcebergHandle`
- `handles.DeltaHandle`
- `handles.LanceHandle`
- `handles.DuckLakeHandle`
- `handles.DoltHandle`
- `handles.MemoryHandle`
- `handles.redact_url`

### Backend protocol

Implement `ObjectBackend` and register it to add a system; capability flags decide which commands apply.

- `backends.ChangeEntry`
- `backends.HistoryEntry`
- `backends.ObjectDiff`
- `backends.VerifyReport`
- `backends.ObjectBackend`
- `backends.Capability`
- `backends.Tier`
- `backends.VerifyStatus`
- `backends.base_at`
- `backends.build_backend`
- `backends.content_state`
- `backends.effective_capabilities`
- `backends.known_kinds`
- `backends.register_backend`
- `backends.tier_of`
- `backends.MAX_DIFF_ENTRIES`
- `backends.Listings`

### Conformance suite

Capability-parametrized checks for backend authors.

- `testing.BackendError`
- `testing.RefMovedError`
- `testing.ObjectDiff`
- `testing.Handle`
- `testing.Pin`
- `testing.Policy`
- `testing.ObjectBackend`
- `testing.BackendHarness`
- `testing.Capability`
- `testing.VerifyStatus`
- `testing.accepts_expected`
- `testing.content_state`
- `testing.effective_capabilities`
- `testing.compute_pin_id`
- `testing.ref_for_pin`
- `testing.working_ref_name`
- `testing.run_conformance`
- `testing.ABSENT`
- `testing.Locator`
- `testing.CONFORMANCE_DATASET`
- `testing.RACERS`

### Registries (experimental)

Export tables (`ExportBundle`, `TABLES`), Postgres publishing, and the canonical import columns / sources for `import`. Lives under `tether.experimental.registry`; import the public names from `tether`.

- `experimental.registry`
- `experimental.registry.ImportReport`

### Store lifecycle (experimental)

`add --create` / `Repo.create` make a store tether owns; `gc --delete-stores` removes it once nothing references it. The created- and touched-store indexes, the planning rules, and the graduation criteria live in `tether.experimental.lifecycle`.

- `experimental.lifecycle`

### VCS adapters

How tether reads and writes history through jj or git.

- `vcs.VcsError`
- `vcs.GitObjectReader`
- `vcs.JjAdapter`
- `vcs.GitAdapter`
- `vcs.CommitInfo`
- `vcs.RefInfo`
- `vcs.VcsAdapter`
- `vcs.git_env`
- `vcs.require_git_version`
- `vcs.detect_vcs`
- `vcs.MIN_JJ_VERSION`
- `vcs.MIN_GIT_VERSION`

### Errors

- `TetherError`: Base class for all tether errors
- `ConfigError`: The repository configuration or manifest layout is invalid
- `VcsError`: A version-control operation failed
- `BackendError`: A backend operation failed
- `CapabilityError`: An operation was requested that the backend does not support
- `ImmutableObjectModified`: An object registered as immutable changed on disk / in storage
- `UnpinnedStateError`: A pinnable object has working state that was never pinned
- `PinDriftError`: A pin no longer resolves to the state recorded in the manifest
- `StaleWorkingCopyError`: The workspace was forked from a manifest that no longer matches HEAD
- `StalePlanError`: A saved plan no longer matches the world it was computed from
- `MergeConflict`: A native merge stopped on conflicts; nothing was written
- `errors.RefMovedError`
- `MultiObjectError`: Aggregates per-object failures from a fan-out operation

## Gotchas

1. A dataset bookmark stands for a branch in every system. The trunk (`main`, `[vcs] trunk`) stands for each object's upstream branch: working on it writes there. `tether new -b NAME` starts a bookmark whose branch in each system is `tether.ws.<dataset>.NAME`, forked on first write. A working copy on no bookmark is read-only: `open --rev`, `status`, `verify` work; writes need a bookmark.
2. `tether status` is local by default and shows the last snapshot with its age; `tether status --snapshot` (or `tether snapshot`) fingerprints every object. `verify` always fingerprints; `commit` fingerprints the bookmark's branches only; objects with no branch keep their pin.
3. `tether commit` moves the bookmark onto the new commit (one jj operation, so `jj undo` takes both back) and leaves the store branches where they are -- it is not `jj commit`, nothing new is forked. It refuses when the VCS working copy has left the bookmark; `status` says how to return.
4. `tether pull` is the fetch: on the trunk it reads every object's upstream branch and every branch-less object (tables, directories) and commits what moved onto `main`. It works only on the bookmark you are on (`tether new NAME` first) and refuses uncommitted manifest edits.
5. Run `tether new` before writing: writable handles are refused while the workspace is stale or has no working ref. Forks are lazy: the first writable `open` creates the branch (`--eager` creates them during `new`; `--pin record` objects always fork during `new`).
6. A bookmark another live checkout works on is refused by `tether new NAME` unless `--shared`; the trunk is shared by nature.
7. A `file` object is Observed unless registered with `--file versioned` on a versioning-enabled bucket; its committed state cannot be re-read later.
8. Locators store local paths absolute, so a clone on another machine cannot reach a local object; pins live in the stores, so a teammate needs store access and an untracked `.tether/secrets.toml` of their own. In a fresh clone run `tether status`, then `tether new NAME` (or `new -b`) before writing.
9. Dolt credentials come only from a `[uris."mysql://host:port"]` entry in `secrets.toml`; `[objects.*]` entries, `[backends.dolt]` and `$DOLT_PASSWORD` by convention are ignored.
10. Neon pins are child branches: a working branch cannot be deleted while its pins exist, and `commit` refuses while writers are active (`--force` overrides).
11. A DuckDB-file DuckLake catalog can be attached once per process; close `DuckLakeHandle`s before `status`.
12. `tether gc` never deletes branches by default, and releases only the pins this clone created (`keep-pin` otherwise; `--release-foreign` opts in). `--prune-bookmarks` deletes a `tether.ws.*` branch of a gone bookmark only if its head is pinned or untouched; `--force-prune` deletes the rest, losing any unpinned data. A branch the store refuses to delete (Neon: pins hang off it) is kept with the reason. `forget-workspace` never touches branches.
13. `tether new` refuses to reset a working branch that holds writes you never committed; commit them or pass `--discard`. A branch already sitting at the pin is reused, not re-forked.
14. `jj undo` / `git reset` move the manifests, never the stores. `tether undo` reverses tether's own operations from its op log (`tether ops`); `tether repair` recreates pins and branches the manifests still name. `status` warns when a dataset commit vanished from the VCS behind tether's back, and when the bookmark was deleted, renamed, moved, or left.
15. A tether command exists where an operation has a VCS half and a store half (`commit`, `new`, `pull`, `promote`, `restore`, `abandon`, `drop`, `forget-workspace`); everything that only touches files and history is jj/git's. Run `tether status` after any VCS operation that moved the working copy or a bookmark, and `tether new` before writing again.

## Best practices

- Set `TETHER_REV` in reproducible jobs so `repo.open(key)` resolves read-only at that commit's pins.
- Production databases that must receive writes on `main` (e.g. Neon, which cannot promote a child branch) are written on the trunk bookmark: `tether new main`.
- `tether promote` lands the bookmark: each system's upstream branch fast-forwards to the bookmark's branch when unchanged since the fork point (a moved upstream needs a backend with `MERGE` -- Dolt, git -- or a manual re-apply; Icechunk/Iceberg refuse, Lance/Neon cannot promote at all), and when everything fast-forwarded the trunk bookmark moves to the commit.
- Run `tether verify --all-history --deep` before `tether gc --no-dry-run`, and as the CI gate: `--deep` is the only way to check Delta, DuckLake and `--pin record` states, which otherwise report `unknown` and exit 1. From CI, promote with `--strategy ff` and write `.tether/secrets.toml` from the runner's secrets before the first command.
- Preview store writes with `--dry-run` on the planned commands (`commit`, `new`, `restore`, `promote`, `gc`, `drop`, `forget-workspace`, `import`, `repair`, `upgrade`); save with `--plan FILE` and apply with `--from-plan FILE` when a review step is needed. `pull`, `undo`, `abandon --gc`, `add --create` and the lazy fork in `open` have no preview.
- Retire a losing bookmark with `tether drop NAME` (dry run first): it leaves the bookmark if the checkout is on it, drops the commits only it reaches, deletes it, and releases its pins and branches -- a branch whose head a dropped commit pinned goes without `--force-prune`. `tether forget-workspace` another checkout that is on it first; `drop` refuses while one is.
- Datasets from any 0.1.0 alpha (before 0.1.0b1) need `tether upgrade` (`--dry-run` first): one migration whose parts run on what the dataset shows. Where pins predate the dataset namespace it renames every pin and branch and rewrites history to match, so every clone must re-sync afterwards; the other parts (content-hashed file states with the `write` policy line dropped; one manifest file per key with absolute local paths) touch only the working tree. `tether.upgrade` ships with the betas and is removed at 0.1.0.
- To hold an input still under a long analysis, work on a bookmark and do not write to it: it keeps its pin. To trail the world, run `tether pull` on `main`.
- Use `--pin record` on scratch objects when only the branch heads matter; it skips the native ref per commit and forks from the recorded state.
- For registries, run `tether publish` after each `tether commit` (or `tether export` for a file); join `objects.uri` to the catalog's URI column and `objects.commit_id` to whatever column records the VCS commit a job read. `tether import` only changes what is tracked; commit afterwards.

## Resources

- [Full documentation](https://rosebud-biosciences.github.io/tether/)
- [llms.txt](llms.txt) — Indexed API reference for LLMs
- [llms-full.txt](llms-full.txt) — Comprehensive documentation for LLMs
- [Source code](https://github.com/Rosebud-Biosciences/tether)