## backends.ObjectBackend The operations tether needs from one class of system. Usage ``` python backends.ObjectBackend() ``` [listing](testing.ObjectBackend.md#tether.testing.ObjectBackend.listing), [diff](testing.ObjectBackend.md#tether.testing.ObjectBackend.diff), and [history](testing.ObjectBackend.md#tether.testing.ObjectBackend.history) have default implementations (no listing; capability errors) so backends without [DIFF](testing.Capability.md#tether.testing.Capability.DIFF) / [HISTORY](testing.Capability.md#tether.testing.Capability.HISTORY) need not define them. Locators may carry an `at` field naming a specific native state (a snapshot id, version, commit, or tag). A backend that supports it treats the object as *detached* there: `fingerprint(locator, None)` returns that state instead of the base branch's head, so [commit](vcs.VcsAdapter.md#tether.vcs.VcsAdapter.commit) pins it and [new](vcs.VcsAdapter.md#tether.vcs.VcsAdapter.new) forks from it. Once a working ref exists it takes precedence. ## Attributes | Name | Description | |----|----| | [KEY_SUFFIXES](#KEY_SUFFIXES) | For a kind whose positional locator ([uri](handles.FileHandle.md#tether.handles.FileHandle.uri)) is a path or URI: the store | | [LOCAL_PATH_KEYS](#LOCAL_PATH_KEYS) | Locator keys whose value may be a local filesystem path. The engine | | [LOCAL_PATH_PREFIXES](#LOCAL_PATH_PREFIXES) | Prefixes a [LOCAL_PATH_KEYS](testing.ObjectBackend.md#tether.testing.ObjectBackend.LOCAL_PATH_KEYS) value may carry before its path | | [MATURITY](#MATURITY) | How much the backend has been exercised: `"stable"` runs its full | | [PROMOTE_HINT](#PROMOTE_HINT) | What to tell a user when tether cannot move the base branch (no | | [SAFE_CONFIG_KEYS](#SAFE_CONFIG_KEYS) | `[backends.]` keys the *committed* `tether.toml` may set. A clone | | [SAFE_OPTION_KEYS](#SAFE_OPTION_KEYS) | For each committed option *table* (`storage_options`, `catalog`), the | | [URI_KEYS](#URI_KEYS) | Locator keys that name the object's store, in order of preference; the | | [VOLATILE_KEYS](#VOLATILE_KEYS) | State keys that *address* the data without identifying it - a Neon LSN | ------------------------------------------------------------------------ #### KEY_SUFFIXES For a kind whose positional locator ([uri](handles.FileHandle.md#tether.handles.FileHandle.uri)) is a path or URI: the store `KEY_SUFFIXES: tuple[str, …] | None = None` suffixes [default_key()](testing.ObjectBackend.md#tether.testing.ObjectBackend.default_key) drops from its last segment (`.icechunk` in `greetings.icechunk`); `()` drops none. `None`: the locator is not a path, and an object of this kind is always registered with a key. ------------------------------------------------------------------------ #### LOCAL_PATH_KEYS Locator keys whose value may be a local filesystem path. The engine `LOCAL_PATH_KEYS: tuple[str, …] = ()` turns a relative one into an absolute path when the object is registered ([add](testing.ObjectDiff.md#tether.testing.ObjectDiff.add), `import`), against the caller's working directory, so a committed locator means the same path from every directory and every clone. ------------------------------------------------------------------------ #### LOCAL_PATH_PREFIXES Prefixes a [LOCAL_PATH_KEYS](testing.ObjectBackend.md#tether.testing.ObjectBackend.LOCAL_PATH_KEYS) value may carry before its path `LOCAL_PATH_PREFIXES: tuple[str, …] = ()` (DuckLake's `ducklake:` or `ducklake:sqlite:`), which would otherwise read as a URL scheme. The path after the longest matching prefix is resolved like a bare one. ------------------------------------------------------------------------ #### MATURITY How much the backend has been exercised: `"stable"` runs its full `MATURITY: str = ``"stable"` lifecycle against the real system in CI (embedded stores and libraries); `"experimental"` is tested against a fake of a network service and has not been run against the service itself by the maintainers. `tether backends` prints it; [add](testing.ObjectDiff.md#tether.testing.ObjectDiff.add) mentions it for experimental kinds. ------------------------------------------------------------------------ #### PROMOTE_HINT What to tell a user when tether cannot move the base branch (no `PROMOTE_HINT: str = ``""` [PROMOTE](testing.Capability.md#tether.testing.Capability.PROMOTE) / [MERGE](testing.Capability.md#tether.testing.Capability.MERGE), or the base diverged and there is no merge). ------------------------------------------------------------------------ #### SAFE_CONFIG_KEYS `[backends.]` keys the *committed* `tether.toml` may set. A clone `SAFE_CONFIG_KEYS: frozenset[str] = frozenset()` arrives with that file, so anything that chooses an executable, an endpoint credentials are sent to, SQL to run, or which environment variable holds a secret must come from the untracked `.tether/secrets.toml` or the environment instead. A key whose value is a table (such as `storage_options`) must also appear in [SAFE_OPTION_KEYS](testing.ObjectBackend.md#tether.testing.ObjectBackend.SAFE_OPTION_KEYS); see `check_committed_config()`. ------------------------------------------------------------------------ #### SAFE_OPTION_KEYS For each committed option *table* (`storage_options`, `catalog`), the `SAFE_OPTION_KEYS: Mapping[str, frozenset[str]] = {}` keys a clone may set inside it. An allowlist, not a pattern: a key the backend has not named is refused, so a new endpoint- or credential-shaped option cannot slip through by spelling. Everything else in that table comes from `secrets.toml`, where the same table may hold any key. ------------------------------------------------------------------------ #### URI_KEYS Locator keys that name the object's store, in order of preference; the `URI_KEYS: tuple[str, …] = (``"uri", ``"path"``)` per-URI entries of `secrets.toml` are matched against the first present. ------------------------------------------------------------------------ #### VOLATILE_KEYS State keys that *address* the data without identifying it - a Neon LSN `VOLATILE_KEYS: frozenset[str] = frozenset()` that advances on checkpoints, a change id derived from a sha. They stay in the state ([pin](testing.ObjectBackend.md#tether.testing.ObjectBackend.pin) and [open](testing.ObjectBackend.md#tether.testing.ObjectBackend.open) need them) but are excluded from equality and from pin ids; see [content_state()](testing.content_state.md#tether.testing.content_state). ## Methods | Name | Description | |----|----| | [ancestor_of()](#ancestor_of) | Whether `ancestor` is in `descendant`'s history (`None`: unknown). | | [base_branch()](#base_branch) | The upstream branch a locator names: what the trunk bookmark stands | | [branch_scope()](#branch_scope) | The native resource that owns branches, as a stable string. | | [configure_cache()](#configure_cache) | Tell the backend where it may keep per-workspace scratch state. | | [configure_checkout()](#configure_checkout) | Tell the backend where the dataset's VCS checkout is. | | [configure_secrets()](#configure_secrets) | Receive the untracked per-checkout settings for this kind. | | [create()](#create) | Make an empty store at `locator` and write the owner marker for | | [default_key()](#default_key) | The key [add](testing.ObjectDiff.md#tether.testing.ObjectDiff.add) gives an object registered without one: the last | | [delete_store()](#delete_store) | Remove the store. Called only after [is_ref_empty()](testing.ObjectBackend.md#tether.testing.ObjectBackend.is_ref_empty) returned | | [delete_working_ref()](#delete_working_ref) | Delete a working ref created by [fork()](testing.ObjectBackend.md#tether.testing.ObjectBackend.fork). Requires [FORK](testing.Capability.md#tether.testing.Capability.FORK). | | [diff()](#diff) | Describe what changed from state `a` to state `b`. Requires [DIFF](testing.Capability.md#tether.testing.Capability.DIFF). | | [fingerprint()](#fingerprint) | Read the current state at `working_ref` (or the locator's base). | | [fork()](#fork) | Create a writable branch `name` off `source`. Requires [FORK](testing.Capability.md#tether.testing.Capability.FORK). | | [history()](#history) | List native history, newest first, starting at [ref](handles.DoltHandle.md#tether.handles.DoltHandle.ref) (default: base). | | [identity()](#identity) | Locator subset that participates in the content-addressed pin id. | | [is_ref_empty()](#is_ref_empty) | Whether nothing remains in the store but its base branch at the | | [list_pins()](#list_pins) | Return pin ids that currently exist natively. Requires [PIN](testing.Capability.md#tether.testing.Capability.PIN). | | [list_working_refs()](#list_working_refs) | Native branches created by [fork()](testing.ObjectBackend.md#tether.testing.ObjectBackend.fork) (`tether.ws.*`). Requires [FORK](testing.Capability.md#tether.testing.Capability.FORK). | | [listing()](#listing) | Optional detailed description of `state` to store alongside it. | | [merge()](#merge) | Three-way merge `source` into the base branch. | | [open()](#open) | Return a native handle for a target. | | [owner()](#owner) | The dataset id in the store's owner marker, or `None` when tether | | [pin()](#pin) | Create a durable native ref for `state`. Requires [PIN](testing.Capability.md#tether.testing.Capability.PIN). | | [promote()](#promote) | Fast-forward the locator's base branch to `source`. Requires [PROMOTE](testing.Capability.md#tether.testing.Capability.PROMOTE). | | [ref_namespace()](#ref_namespace) | The native resource whose pins [list_pins](testing.ObjectBackend.md#tether.testing.ObjectBackend.list_pins) returns, as a string. | | [rename_pin()](#rename_pin) | Give the pin `old` (which names `state`) the id `new_id`. | | [rename_working_ref()](#rename_working_ref) | Rename working branch `old` to [new](vcs.VcsAdapter.md#tether.vcs.VcsAdapter.new); return the resulting ref. | | [secret_rule_for()](#secret_rule_for) | The one object's own `secrets.toml` entry - the longest matching | | [secrets_for()](#secrets_for) | Credential options for one object: the longest matching URI prefix | | [state_addressable()](#state_addressable) | Whether *this* recorded state can be opened again later. | | [unpin()](#unpin) | Release a pin. Requires [PIN](testing.Capability.md#tether.testing.Capability.PIN). | | [validate_locator()](#validate_locator) | Refuse a locator this backend must never be handed. | | [verify()](#verify) | Check that `state`/[pin](testing.ObjectBackend.md#tether.testing.ObjectBackend.pin) still hold. | | [working_ref_blockers()](#working_ref_blockers) | Why [ref](handles.DoltHandle.md#tether.handles.DoltHandle.ref) cannot be deleted right now, or `None` if it can. | ------------------------------------------------------------------------ #### ancestor_of() Whether `ancestor` is in `descendant`'s history (`None`: unknown). Usage ``` python ancestor_of(locator, ancestor, descendant) ``` Used by [promote](testing.ObjectBackend.md#tether.testing.ObjectBackend.promote) when no fork point was recorded. Default: unknown. ------------------------------------------------------------------------ #### base_branch() The upstream branch a locator names: what the trunk bookmark stands Usage ``` python base_branch(locator) ``` for, [promote](testing.ObjectBackend.md#tether.testing.ObjectBackend.promote) moves, and a trunk working copy writes to. Default `locator["branch"]` (`main`); git uses the checked-out branch when the locator has no [ref](handles.DoltHandle.md#tether.handles.DoltHandle.ref). ------------------------------------------------------------------------ #### branch_scope() The native resource that owns branches, as a stable string. Usage ``` python branch_scope(locator) ``` Two objects with the same scope share one branch namespace: a fork under a bookmark creates *one* branch for both, and the engine forks it once and lets every member write through it. Defaults to the canonical identity (one object per system). Override where several objects legitimately live in one branch space - a Neon project, an Iceberg table with several object keys - and where the identity carries fields the branch does not (a database, a source branch). ------------------------------------------------------------------------ #### configure_cache() Tell the backend where it may keep per-workspace scratch state. Usage ``` python configure_cache(cache_dir) ``` Called by the engine after construction with `.tether/cache/` (untracked). Backends that remember expensive results between runs - the `file` backend keeps a stat -\> content-hash cache there - store them under this directory; everything else ignores the call. ------------------------------------------------------------------------ #### configure_checkout() Tell the backend where the dataset's VCS checkout is. Usage ``` python configure_checkout(root) ``` Called by the engine after construction. Every file under `root` arrived with the clone, so a backend that would *run* something a directory there configures - the `git` backend reads a repository's config - refuses paths inside it; everything else ignores the call. ------------------------------------------------------------------------ #### configure_secrets() Receive the untracked per-checkout settings for this kind. Usage ``` python configure_secrets(defaults, rules) ``` `defaults` is the merged `[backends.]` (committed allowlisted keys under the secrets file's); `rules` maps a URI prefix to the credential options for objects under it (the engine folds `[objects.""]` entries in as exact-URI rules). Backends read them through [secrets_for()](testing.ObjectBackend.md#tether.testing.ObjectBackend.secrets_for). ------------------------------------------------------------------------ #### create() Make an empty store at `locator` and write the owner marker for Usage ``` python create(locator, *, owner) ``` [owner](testing.ObjectBackend.md#tether.testing.ObjectBackend.owner) (a dataset id); return its initial state. Requires [CREATE](testing.Capability.md#tether.testing.Capability.CREATE). ##### Raises `BackendError` Anything already exists there - [create](testing.ObjectBackend.md#tether.testing.ObjectBackend.create) never adopts a store someone else made. ------------------------------------------------------------------------ #### default_key() The key [add](testing.ObjectDiff.md#tether.testing.ObjectDiff.add) gives an object registered without one: the last Usage ``` python default_key(locator) ``` segment of the locator's [uri](handles.FileHandle.md#tether.handles.FileHandle.uri) (or [path](handles.GitHandle.md#tether.handles.GitHandle.path)), less one of [KEY_SUFFIXES](testing.ObjectBackend.md#tether.testing.ObjectBackend.KEY_SUFFIXES). `None` when this kind names no path or no valid key comes out. ------------------------------------------------------------------------ #### delete_store() Remove the store. Called only after [is_ref_empty()](testing.ObjectBackend.md#tether.testing.ObjectBackend.is_ref_empty) returned Usage ``` python delete_store(locator) ``` `True` for it, and only for a store whose [owner()](testing.ObjectBackend.md#tether.testing.ObjectBackend.owner) is the caller's dataset - and expected to check both again itself, right before the irreversible step. Requires [CREATE](testing.Capability.md#tether.testing.Capability.CREATE). ------------------------------------------------------------------------ #### delete_working_ref() Delete a working ref created by [fork()](testing.ObjectBackend.md#tether.testing.ObjectBackend.fork). Requires [FORK](testing.Capability.md#tether.testing.Capability.FORK). Usage ``` python delete_working_ref(locator, ref) ``` ------------------------------------------------------------------------ #### diff() Describe what changed from state `a` to state `b`. Requires [DIFF](testing.Capability.md#tether.testing.Capability.DIFF). Usage ``` python diff(locator, a, b, *, listings=(None, None)) ``` ------------------------------------------------------------------------ #### fingerprint() Read the current state at `working_ref` (or the locator's base). Usage ``` python fingerprint(locator, working_ref) ``` ------------------------------------------------------------------------ #### fork() Create a writable branch `name` off `source`. Requires [FORK](testing.Capability.md#tether.testing.Capability.FORK). Usage ``` python fork(locator, source, name, *, expected=None) ``` `source` is a `~tether.manifest.Pin` (fork from the native ref) or a recorded `State` (fork directly from an addressable state, used by `policy.pin == "record"` objects that carry no native ref). **Reset contract**: if `name` already exists it is moved back onto `source` (whatever was written on it is discarded); a branch already at `source` is left alone. The conformance suite checks this. **Conditional move**: `expected` is the head `name` must hold for the move to happen - the state the caller reviewed, or [ABSENT](testing.ABSENT.md#tether.testing.ABSENT) when the branch must not exist yet. When it holds something else, raise `~tether.errors.RefMovedError` and move nothing (compare content states: [VOLATILE_KEYS](testing.ObjectBackend.md#tether.testing.ObjectBackend.VOLATILE_KEYS) may differ). `None` moves unconditionally, as before 0.1.0b4; see [accepts_expected()](testing.accepts_expected.md#tether.testing.accepts_expected) for how the engine treats a backend whose signature predates the keyword. ------------------------------------------------------------------------ #### history() List native history, newest first, starting at [ref](handles.DoltHandle.md#tether.handles.DoltHandle.ref) (default: base). Usage ``` python history(locator, ref=None, limit=20) ``` Requires [HISTORY](testing.Capability.md#tether.testing.Capability.HISTORY). Metadata only; `limit` bounds the walk. ------------------------------------------------------------------------ #### identity() Locator subset that participates in the content-addressed pin id. Usage ``` python identity(locator) ``` Defaults to the full locator; override to exclude non-identity fields (region, credential references, source branch, …). ------------------------------------------------------------------------ #### is_ref_empty() Whether nothing remains in the store but its base branch at the Usage ``` python is_ref_empty(locator, *, ignoring=()) ``` initial state, the owner marker, and the refs in `ignoring` (the ones a gc plan is about to delete). A store with a working area (a git checkout, a directory) must also hold no uncommitted content: files are data whether or not a ref names them. Requires [CREATE](testing.Capability.md#tether.testing.Capability.CREATE). `None` means the backend cannot tell - and `None` means *keep*: a store is only ever removed on a definite `True`. ------------------------------------------------------------------------ #### list_pins() Return pin ids that currently exist natively. Requires [PIN](testing.Capability.md#tether.testing.Capability.PIN). Usage ``` python list_pins(locator) ``` ------------------------------------------------------------------------ #### list_working_refs() Native branches created by [fork()](testing.ObjectBackend.md#tether.testing.ObjectBackend.fork) (`tether.ws.*`). Requires [FORK](testing.Capability.md#tether.testing.Capability.FORK). Usage ``` python list_working_refs(locator) ``` Default: none. Used by `gc --prune-bookmarks`. ------------------------------------------------------------------------ #### listing() Optional detailed description of `state` to store alongside it. Usage ``` python listing(locator, state) ``` The engine writes the text content-addressed under `.tether/listings/` at commit time and hands both sides back to [diff()](testing.ObjectBackend.md#tether.testing.ObjectBackend.diff) later. Used by backends whose state is a digest (the `file` backend's directory and prefix listings) so Observed states can still be diffed. ------------------------------------------------------------------------ #### merge() Three-way merge `source` into the base branch. Usage ``` python merge(locator, source, message, *, expected=None) ``` `source` is a working ref, a [Pin](Pin.md#tether.Pin), or a `State`. The engine passes the *state* the plan reviewed, so what is merged is what was shown - a ref is a name someone can move between plan and apply. Requires [MERGE](testing.Capability.md#tether.testing.Capability.MERGE). Raises `~tether.errors.MergeConflict` (and leaves the base untouched) when the system reports conflicts, and `~tether.errors.RefMovedError` when `expected` is given and the base head is not that state. Returns the base branch's new state. ------------------------------------------------------------------------ #### open() Return a native handle for a target. Usage ``` python open(locator, target, read_only) ``` [target](Action.md#tether.Action.target) is one of: `None` (the base/working ref), a `str` working ref, a `~tether.manifest.Pin` (read a pinned state), or a recorded `State` mapping (read an addressable state without a pin). ------------------------------------------------------------------------ #### owner() The dataset id in the store's owner marker, or `None` when tether Usage ``` python owner(locator) ``` never wrote one (or the store is gone). Requires [CREATE](testing.Capability.md#tether.testing.Capability.CREATE). ------------------------------------------------------------------------ #### pin() Create a durable native ref for `state`. Requires [PIN](testing.Capability.md#tether.testing.Capability.PIN). Usage ``` python pin(locator, state, pin_id) ``` ------------------------------------------------------------------------ #### promote() Fast-forward the locator's base branch to `source`. Requires [PROMOTE](testing.Capability.md#tether.testing.Capability.PROMOTE). Usage ``` python promote(locator, source, *, expected=None) ``` `source` is a working ref name, a `~tether.manifest.Pin`, or a recorded `State`. Implementations must refuse ([BackendError](BackendError.md#tether.BackendError)) when the base head is not an ancestor of `source` - a fast-forward never discards anything - and be a no-op when the base is already there. `expected` is the base head the caller reviewed: when the base holds another state, raise `~tether.errors.RefMovedError` and move nothing (a compare-and-swap where the system has one - git's `update-ref` with an old value, Icechunk's `from_snapshot_id` on an object store; see [Capability.CONDITIONAL_REF](testing.Capability.md#tether.testing.Capability.CONDITIONAL_REF)). Returns the base branch's new state. ------------------------------------------------------------------------ #### ref_namespace() The native resource whose pins [list_pins](testing.ObjectBackend.md#tether.testing.ObjectBackend.list_pins) returns, as a string. Usage ``` python ref_namespace(locator) ``` `gc` compares the pins it lists against the pins every object *in that namespace* references; listing a project-wide namespace against one object's references would release the others' pins. Defaults to the canonical identity. ------------------------------------------------------------------------ #### rename_pin() Give the pin `old` (which names `state`) the id `new_id`. Usage ``` python rename_pin(locator, old, state, new_id) ``` Used by `tether upgrade` when the pin naming scheme changes. Default: [pin()](testing.ObjectBackend.md#tether.testing.ObjectBackend.pin) the state under the new id, then [unpin()](testing.ObjectBackend.md#tether.testing.ObjectBackend.unpin) the old one - which works wherever pins are tags. Backends whose pins are branches with children (Neon) rename in place instead. ------------------------------------------------------------------------ #### rename_working_ref() Rename working branch `old` to [new](vcs.VcsAdapter.md#tether.vcs.VcsAdapter.new); return the resulting ref. Usage ``` python rename_working_ref(locator, old, new) ``` Used by `tether upgrade` when the branch naming scheme changes. Default: fork [new](vcs.VcsAdapter.md#tether.vcs.VcsAdapter.new) from `old`'s head state and delete `old`. Requires [FORK](testing.Capability.md#tether.testing.Capability.FORK) and a backend that can fork from a state. ------------------------------------------------------------------------ #### secret_rule_for() The one object's own `secrets.toml` entry - the longest matching Usage ``` python secret_rule_for(locator) ``` URI-prefix rule, without the kind defaults [secrets_for](testing.ObjectBackend.md#tether.testing.ObjectBackend.secrets_for) layers it on; empty when none matches. ------------------------------------------------------------------------ #### secrets_for() Credential options for one object: the longest matching URI prefix Usage ``` python secrets_for(locator) ``` rule over the kind defaults; empty when nothing applies (the backend then falls back to the environment, as it always did). ------------------------------------------------------------------------ #### state_addressable() Whether *this* recorded state can be opened again later. Usage ``` python state_addressable(locator, state) ``` The capability says the backend can address states in general; a particular fingerprint may still lack the coordinate (an object in a bucket without versioning has no [version_id](handles.FileHandle.md#tether.handles.FileHandle.version_id)). [commit](vcs.VcsAdapter.md#tether.vcs.VcsAdapter.commit) records such a state as not recoverable instead of promising a read it cannot do. ------------------------------------------------------------------------ #### unpin() Release a pin. Requires [PIN](testing.Capability.md#tether.testing.Capability.PIN). Usage ``` python unpin(locator, pin) ``` ------------------------------------------------------------------------ #### validate_locator() Refuse a locator this backend must never be handed. Usage ``` python validate_locator(locator) ``` Called when an object is registered ([add](testing.ObjectDiff.md#tether.testing.ObjectDiff.add), `import`) so the refusal comes with the manifest's author present, not at the first read of a clone. Default: accept. Raise [BackendError](BackendError.md#tether.BackendError) to refuse. ------------------------------------------------------------------------ #### verify() Check that `state`/[pin](testing.ObjectBackend.md#tether.testing.ObjectBackend.pin) still hold. Usage ``` python verify(locator, state, pin, deep) ``` ------------------------------------------------------------------------ #### working_ref_blockers() Why [ref](handles.DoltHandle.md#tether.handles.DoltHandle.ref) cannot be deleted right now, or `None` if it can. Usage ``` python working_ref_blockers(locator, ref) ``` Consulted by `gc --prune-bookmarks` and `forget-workspace` before planning a `delete-branch`, so an undeletable branch is planned as kept with the reason instead of failing at apply time. Neon: a branch with children (pins taken on it) cannot be deleted until they are. Default: no blockers.