Skip to content

Database (async/sync)

AsyncDatabase and Database share one API, generated from a single async source via unasync (see docs/testing.md) — every method below exists identically on both, async/ await aside.

The .raw escape hatch — you now own error handling

Every connection/transaction scope (returned by db.connection()/db.transaction()) exposes a .raw property: the underlying SQLAlchemy Connection/AsyncConnection. It exists for the two things dbkit doesn't implement itself — tx.pipeline() and db.copy_records()'s raw driver access both use it internally — and for genuine one-off raw-driver needs.

Using .raw directly opts you out of dbkit's entire value proposition for that statement: no SQLSTATE classification (you get raw psycopg/asyncpg/SQLAlchemy exceptions, not DatabaseError subclasses — an except DatabaseError handler will silently stop catching them), no metrics, no tracing, no retry/circuit-breaker/commit-unknown handling. This is by design (dbkit can't classify or retry a statement it never saw), but it's easy to reach for as a shortcut and not notice you've silently disabled error handling for that code path. Prefer the normal db.execute/fetch_*/stream/bulk methods for anything that isn't one of the two documented raw-driver escape hatches.

Advisory locks (transaction scope)

The scope from db.transaction(...) also exposes transaction-scoped PostgreSQL advisory locks — cooperative locks on a logical key (an order id, an engagement id) rather than on rows:

async with db.transaction(target=t, lock_timeout=3.0) as tx:
    await tx.advisory_xact_lock("engagement:42")   # blocks until granted; auto-released at commit
    ...                                            # read-modify-write, serialized per key

await tx.advisory_xact_lock(key) blocks until granted and holds until the transaction ends (there is no manual unlock — it can't leak); a wait beyond the transaction's lock_timeout raises DatabaseLockTimeoutError. await tx.try_advisory_xact_lock(key) -> bool is the non-blocking variant (False if another transaction holds it). key is an int (used directly) or a str (hashed server-side); it is always bound, never interpolated. PostgreSQL only.

AsyncDatabase

AsyncDatabase(
    config: DbkitConfig,
    *,
    metrics: MetricsSink | None = None,
    tracer: Tracer | None = None,
    shard_resolver: ShardResolver | None = None,
    replica_selector: ReplicaSelector | None = None,
)

Async, SQL-first database facade over SQLAlchemy Core (§8.1).

Build a facade from an already-validated :class:DbkitConfig.

Prefer :meth:from_config unless you need to pass a config object you built/validated yourself. Does not connect to any database — call :meth:start before use.

config property

config: DbkitConfig

The validated configuration this facade was built from.

engine_count property

engine_count: int

Number of live SQLAlchemy engines currently held (one per target).

from_config classmethod

from_config(
    config: DbkitConfig | Mapping[str, Any],
    *,
    metrics: MetricsSink | None = None,
    tracer: Tracer | None = None,
    shard_resolver: ShardResolver | None = None,
    replica_selector: ReplicaSelector | None = None,
) -> AsyncDatabase

Build a facade from a :class:DbkitConfig or a plain dict (validated via :meth:DbkitConfig.from_dict). The usual entry point.

start async

start(*, warm: bool = False) -> None

Create required engines and (optionally) warm connections (§27.1).

require_ready async

require_ready() -> None

Raise unless every required target is reachable (§27.1).

close async

close(grace_period: float = 10.0) -> None

Dispose all engines (§27.2). grace_period reserved for in-flight draining.

register_database async

register_database(
    name: str,
    config: DatabaseConfig | Mapping[str, Any],
    *,
    connect: bool | None = None,
) -> bool

Register (or replace) a named database at runtime.

For topologies discovered after :meth:start — e.g. shard DSNs resolved per tenant from a service registry — so callers do not need to maintain their own engine registries. config is a :class:DatabaseConfig or the same dict shape as one entry under the top-level databases mapping.

The database map is swapped copy-on-write, so concurrent readers always see a complete mapping. Replacing an existing name first disposes its engines and resets its limiter/breakers, so the new settings fully apply. Returns True when an existing entry was replaced.

connect forces (or suppresses) eager engine creation; the default follows primary.required when the facade is already started. Engine creation is lazy in SQLAlchemy, so this never blocks on the network unless the pool is warmed explicitly.

ensure_database async

ensure_database(
    name: str,
    config: DatabaseConfig | Mapping[str, Any],
    *,
    connect: bool | None = None,
) -> bool

Idempotent :meth:register_database: act only when name is missing or its config changed. Returns True when a (re)registration actually happened.

The unchanged path is lock-free and cheap (build + compare one frozen config, a few µs), so callers routing by runtime topology can call this in front of every query::

await db.ensure_database(shard_id, {"primary": {"url": dsn}})
rows = await db.fetch_all(query, params,
                          target=DatabaseTarget(database=shard_id, role="read"))

A changed config (rotated host, credentials, pool settings) re-registers in place: old engines are disposed and the next query connects with the new settings — no restart needed, including password-only rotations.

unregister_database async

unregister_database(name: str) -> bool

Remove a dynamically (or statically) registered database and dispose its engines, limiter, and breakers. Returns False if name is unknown.

In-flight operations against the database finish on their checked-out connections (idle-only disposal); new calls raise DatabaseRoutingError.

database_scope async

database_scope(
    name: str,
    config: DatabaseConfig | Mapping[str, Any],
    *,
    connect: bool | None = None,
) -> AsyncIterator[DatabaseTarget]

Register name for the duration of the block, then unregister it.

Yields a primary-routed :class:DatabaseTarget for convenience::

async with db.database_scope("tenant-42", {"primary": {"url": dsn}}) as target:
    rows = await db.fetch_all(query, params, target=target)

Scope this to a unit of work, never to a request. Engines/pools are created on first use inside the block and disposed on exit — wrapping every HTTP request in a scope recreates connections each time and defeats pooling entirely. Intended uses: tests, migrations, one-off tenant batch jobs, or an ad-hoc query against a shard the process does not normally serve. Long-lived services should call :meth:register_database once per discovered shard and keep it registered.

fetch_all async

fetch_all(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    map_to: Any = None,
    timeout: float | None = None,
    deadline: float | None = None,
) -> list[Any]

Run a one-shot read, acquiring and releasing its own connection (§11.1).

Returns every row, mapped to map_to, with no cardinality constraint. Retries and the circuit breaker apply per the resolved target's policy.

fetch_one async

fetch_one(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    map_to: Any = None,
    timeout: float | None = None,
    deadline: float | None = None,
) -> Any

Run a one-shot read expecting exactly one row.

Raises :class:~dbkit.errors.DatabaseResultError if zero or more than one row comes back (via SQLAlchemy's own Result.one()).

fetch_optional async

fetch_optional(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    map_to: Any = None,
    timeout: float | None = None,
    deadline: float | None = None,
) -> Any | None

Run a one-shot read expecting zero or one row; None if empty.

fetch_value async

fetch_value(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    timeout: float | None = None,
    deadline: float | None = None,
) -> Any

Run a one-shot read expecting exactly one row and return its first column.

fetch_values async

fetch_values(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    timeout: float | None = None,
    deadline: float | None = None,
) -> list[Any]

Run a one-shot read and return the first column of every row.

execute async

execute(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    timeout: float | None = None,
    deadline: float | None = None,
) -> ExecutionResult

Run a one-shot write/DDL statement, auto-committed on success (§11.1).

execute_many async

execute_many(
    query: object,
    params_seq: Sequence[Mapping[str, Any]],
    *,
    target: DatabaseTarget,
    timeout: float | None = None,
    deadline: float | None = None,
) -> ExecutionResult

Run one statement against many parameter sets (driver executemany).

connection async

connection(
    *, target: DatabaseTarget, timeout: float | None = None
) -> AsyncIterator[AsyncConnectionScope]

A held connection with commit-on-success / rollback-on-error semantics (§11.2).

transaction async

transaction(
    *,
    target: DatabaseTarget,
    isolation: str | None = None,
    read_only: bool = False,
    deferrable: bool = False,
    timeout: float | None = None,
    lock_timeout: float | None = None,
) -> AsyncIterator[Any]

An explicit transaction with isolation/timeout options (§11.3).

deferrable (PostgreSQL only) is only meaningful together with isolation="serializable" and read_only=True: it lets a read-only serializable transaction wait for a safe snapshot instead of risking a serialization failure.

Usage::

async with db.transaction(target=write_target) as tx:
    await tx.execute(INSERT, params)

consistency_scope async

consistency_scope(
    *,
    mode: Literal["read_your_writes"] = "read_your_writes",
) -> AsyncIterator[None]

Force reads within this scope to the primary so they observe writes made earlier in the same scope — read-your-writes over replica routing (§23)::

async with db.consistency_scope(mode="read_your_writes"):
    await db.execute(write_query, target=write_target)
    row = await db.fetch_one(read_query, target=read_target)  # sees the write

The override is a contextvars.ContextVar, so it never leaks across concurrent operations in other tasks — but it also does not cross a thread boundary (a plain threading.Thread, loop.run_in_executor, or the sync Database facade driven from a worker thread does not inherit it). See "Read-your-writes across threads" in docs/troubleshooting.md if you need the override to apply there too.

health async

health() -> HealthReport

Readiness check: SELECT 1 against every required target (§26).

pool_status

pool_status() -> list[PoolSnapshot]

A point-in-time snapshot of every currently live engine's connection pool.

drain_engine async

drain_engine(key: str) -> bool

Force-dispose one engine's idle pooled connections by its snapshot key (as printed by :meth:pool_status, e.g. "prod:app:default:primary:psycopg"), so the next call routed to it rebuilds a fresh engine with fresh connections. Returns False if no live engine currently has that key.

Useful right before a planned failover/topology change: rather than waiting for connections to naturally recycle (PoolConfig.recycle_seconds), force them closed so subsequent traffic reaches the new backend immediately. Only idle pooled connections are closed — one already checked out by an in-flight call keeps working until released, exactly like LRU eviction (AsyncEngineRegistry(evict_lru=True)).

This must be called from within the running application process — there is no dbkit CLI command for this, deliberately: each CLI invocation is a fresh, separate process with its own empty engine registry, so it has no way to reach into an already-running application's live engines. Wire this to a signal handler or an admin HTTP route in your own application instead (see docs/troubleshooting.md).

stream async

stream(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    batch_size: int = 1000,
    map_to: Any = None,
    max_duration: float | None = None,
) -> AsyncResultStream

Stream a large result set with a server-side cursor, bounded memory (§20).

Usage::

async with await db.stream(EXPORT, params, target=t, batch_size=1000) as rows:
    async for row in rows:
        ...

The stream owns its connection until the context exits, so it bypasses auto-retry.

insert_many async

insert_many(
    table: Table,
    rows: Sequence[Mapping[str, Any]],
    *,
    target: DatabaseTarget,
    mode: FailureMode = "atomic",
    strategy: InsertStrategy = "execute_many",
    batch_size: int | None = None,
    timeout: float | None = None,
) -> ExecutionResult

Insert many rows in adaptively-sized batches (§19). rows are param dicts.

strategy="unnest" (PostgreSQL only) binds one array parameter per column instead of one parameter per column per row, so batch size isn't limited by the bind-parameter ceiling — a mid-tier option between execute_many and COPY.

upsert_many async

upsert_many(
    table: Table,
    rows: Sequence[Mapping[str, Any]],
    *,
    target: DatabaseTarget,
    conflict_index_elements: Sequence[str],
    update_columns: Sequence[str] | None = None,
    mode: FailureMode = "atomic",
    strategy: InsertStrategy = "execute_many",
    batch_size: int | None = None,
    timeout: float | None = None,
) -> ExecutionResult

Upsert many rows via PostgreSQL ON CONFLICT (§19). update_columns=None means DO NOTHING; otherwise those columns are updated from the proposed row.

copy_records async

copy_records(
    table: str,
    columns: Sequence[str],
    records: Any,
    *,
    target: DatabaseTarget,
    timeout: float | None = None,
) -> ExecutionResult

Bulk-ingest records into table via PostgreSQL COPY (§19.2).

records is an iterable (async or sync) of row sequences matching columns. Fastest path for large ingests; PostgreSQL + psycopg only.

Database

Database(
    config: DbkitConfig,
    *,
    metrics: MetricsSink | None = None,
    tracer: Tracer | None = None,
    shard_resolver: ShardResolver | None = None,
    replica_selector: ReplicaSelector | None = None,
)

Async, SQL-first database facade over SQLAlchemy Core (§8.1).

Build a facade from an already-validated :class:DbkitConfig.

Prefer :meth:from_config unless you need to pass a config object you built/validated yourself. Does not connect to any database — call :meth:start before use.

config property

config: DbkitConfig

The validated configuration this facade was built from.

engine_count property

engine_count: int

Number of live SQLAlchemy engines currently held (one per target).

from_config classmethod

from_config(
    config: DbkitConfig | Mapping[str, Any],
    *,
    metrics: MetricsSink | None = None,
    tracer: Tracer | None = None,
    shard_resolver: ShardResolver | None = None,
    replica_selector: ReplicaSelector | None = None,
) -> Database

Build a facade from a :class:DbkitConfig or a plain dict (validated via :meth:DbkitConfig.from_dict). The usual entry point.

start

start(*, warm: bool = False) -> None

Create required engines and (optionally) warm connections (§27.1).

require_ready

require_ready() -> None

Raise unless every required target is reachable (§27.1).

close

close(grace_period: float = 10.0) -> None

Dispose all engines (§27.2). grace_period reserved for in-flight draining.

register_database

register_database(
    name: str,
    config: DatabaseConfig | Mapping[str, Any],
    *,
    connect: bool | None = None,
) -> bool

Register (or replace) a named database at runtime.

For topologies discovered after :meth:start — e.g. shard DSNs resolved per tenant from a service registry — so callers do not need to maintain their own engine registries. config is a :class:DatabaseConfig or the same dict shape as one entry under the top-level databases mapping.

The database map is swapped copy-on-write, so concurrent readers always see a complete mapping. Replacing an existing name first disposes its engines and resets its limiter/breakers, so the new settings fully apply. Returns True when an existing entry was replaced.

connect forces (or suppresses) eager engine creation; the default follows primary.required when the facade is already started. Engine creation is lazy in SQLAlchemy, so this never blocks on the network unless the pool is warmed explicitly.

ensure_database

ensure_database(
    name: str,
    config: DatabaseConfig | Mapping[str, Any],
    *,
    connect: bool | None = None,
) -> bool

Idempotent :meth:register_database: act only when name is missing or its config changed. Returns True when a (re)registration actually happened.

The unchanged path is lock-free and cheap (build + compare one frozen config, a few µs), so callers routing by runtime topology can call this in front of every query::

db.ensure_database(shard_id, {"primary": {"url": dsn}})
rows = db.fetch_all(query, params,
                          target=DatabaseTarget(database=shard_id, role="read"))

A changed config (rotated host, credentials, pool settings) re-registers in place: old engines are disposed and the next query connects with the new settings — no restart needed, including password-only rotations.

unregister_database

unregister_database(name: str) -> bool

Remove a dynamically (or statically) registered database and dispose its engines, limiter, and breakers. Returns False if name is unknown.

In-flight operations against the database finish on their checked-out connections (idle-only disposal); new calls raise DatabaseRoutingError.

database_scope

database_scope(
    name: str,
    config: DatabaseConfig | Mapping[str, Any],
    *,
    connect: bool | None = None,
) -> Iterator[DatabaseTarget]

Register name for the duration of the block, then unregister it.

Yields a primary-routed :class:DatabaseTarget for convenience::

with db.database_scope("tenant-42", {"primary": {"url": dsn}}) as target:
    rows = db.fetch_all(query, params, target=target)

Scope this to a unit of work, never to a request. Engines/pools are created on first use inside the block and disposed on exit — wrapping every HTTP request in a scope recreates connections each time and defeats pooling entirely. Intended uses: tests, migrations, one-off tenant batch jobs, or an ad-hoc query against a shard the process does not normally serve. Long-lived services should call :meth:register_database once per discovered shard and keep it registered.

fetch_all

fetch_all(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    map_to: Any = None,
    timeout: float | None = None,
    deadline: float | None = None,
) -> list[Any]

Run a one-shot read, acquiring and releasing its own connection (§11.1).

Returns every row, mapped to map_to, with no cardinality constraint. Retries and the circuit breaker apply per the resolved target's policy.

fetch_one

fetch_one(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    map_to: Any = None,
    timeout: float | None = None,
    deadline: float | None = None,
) -> Any

Run a one-shot read expecting exactly one row.

Raises :class:~dbkit.errors.DatabaseResultError if zero or more than one row comes back (via SQLAlchemy's own Result.one()).

fetch_optional

fetch_optional(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    map_to: Any = None,
    timeout: float | None = None,
    deadline: float | None = None,
) -> Any | None

Run a one-shot read expecting zero or one row; None if empty.

fetch_value

fetch_value(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    timeout: float | None = None,
    deadline: float | None = None,
) -> Any

Run a one-shot read expecting exactly one row and return its first column.

fetch_values

fetch_values(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    timeout: float | None = None,
    deadline: float | None = None,
) -> list[Any]

Run a one-shot read and return the first column of every row.

execute

execute(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    timeout: float | None = None,
    deadline: float | None = None,
) -> ExecutionResult

Run a one-shot write/DDL statement, auto-committed on success (§11.1).

execute_many

execute_many(
    query: object,
    params_seq: Sequence[Mapping[str, Any]],
    *,
    target: DatabaseTarget,
    timeout: float | None = None,
    deadline: float | None = None,
) -> ExecutionResult

Run one statement against many parameter sets (driver executemany).

connection

connection(
    *, target: DatabaseTarget, timeout: float | None = None
) -> Iterator[ConnectionScope]

A held connection with commit-on-success / rollback-on-error semantics (§11.2).

transaction

transaction(
    *,
    target: DatabaseTarget,
    isolation: str | None = None,
    read_only: bool = False,
    deferrable: bool = False,
    timeout: float | None = None,
    lock_timeout: float | None = None,
) -> Iterator[Any]

An explicit transaction with isolation/timeout options (§11.3).

deferrable (PostgreSQL only) is only meaningful together with isolation="serializable" and read_only=True: it lets a read-only serializable transaction wait for a safe snapshot instead of risking a serialization failure.

Usage::

with db.transaction(target=write_target) as tx:
    tx.execute(INSERT, params)

consistency_scope

consistency_scope(
    *,
    mode: Literal["read_your_writes"] = "read_your_writes",
) -> Iterator[None]

Force reads within this scope to the primary so they observe writes made earlier in the same scope — read-your-writes over replica routing (§23)::

with db.consistency_scope(mode="read_your_writes"):
    db.execute(write_query, target=write_target)
    row = db.fetch_one(read_query, target=read_target)  # sees the write

The override is a contextvars.ContextVar, so it never leaks across concurrent operations in other tasks — but it also does not cross a thread boundary (a plain threading.Thread, loop.run_in_executor, or the sync Database facade driven from a worker thread does not inherit it). See "Read-your-writes across threads" in docs/troubleshooting.md if you need the override to apply there too.

health

health() -> HealthReport

Readiness check: SELECT 1 against every required target (§26).

pool_status

pool_status() -> list[PoolSnapshot]

A point-in-time snapshot of every currently live engine's connection pool.

drain_engine

drain_engine(key: str) -> bool

Force-dispose one engine's idle pooled connections by its snapshot key (as printed by :meth:pool_status, e.g. "prod:app:default:primary:psycopg"), so the next call routed to it rebuilds a fresh engine with fresh connections. Returns False if no live engine currently has that key.

Useful right before a planned failover/topology change: rather than waiting for connections to naturally recycle (PoolConfig.recycle_seconds), force them closed so subsequent traffic reaches the new backend immediately. Only idle pooled connections are closed — one already checked out by an in-flight call keeps working until released, exactly like LRU eviction (EngineRegistry(evict_lru=True)).

This must be called from within the running application process — there is no dbkit CLI command for this, deliberately: each CLI invocation is a fresh, separate process with its own empty engine registry, so it has no way to reach into an already-running application's live engines. Wire this to a signal handler or an admin HTTP route in your own application instead (see docs/troubleshooting.md).

stream

stream(
    query: object,
    params: Mapping[str, Any] | None = None,
    *,
    target: DatabaseTarget,
    batch_size: int = 1000,
    map_to: Any = None,
    max_duration: float | None = None,
) -> ResultStream

Stream a large result set with a server-side cursor, bounded memory (§20).

Usage::

with db.stream(EXPORT, params, target=t, batch_size=1000) as rows:
    for row in rows:
        ...

The stream owns its connection until the context exits, so it bypasses auto-retry.

insert_many

insert_many(
    table: Table,
    rows: Sequence[Mapping[str, Any]],
    *,
    target: DatabaseTarget,
    mode: FailureMode = "atomic",
    strategy: InsertStrategy = "execute_many",
    batch_size: int | None = None,
    timeout: float | None = None,
) -> ExecutionResult

Insert many rows in adaptively-sized batches (§19). rows are param dicts.

strategy="unnest" (PostgreSQL only) binds one array parameter per column instead of one parameter per column per row, so batch size isn't limited by the bind-parameter ceiling — a mid-tier option between execute_many and COPY.

upsert_many

upsert_many(
    table: Table,
    rows: Sequence[Mapping[str, Any]],
    *,
    target: DatabaseTarget,
    conflict_index_elements: Sequence[str],
    update_columns: Sequence[str] | None = None,
    mode: FailureMode = "atomic",
    strategy: InsertStrategy = "execute_many",
    batch_size: int | None = None,
    timeout: float | None = None,
) -> ExecutionResult

Upsert many rows via PostgreSQL ON CONFLICT (§19). update_columns=None means DO NOTHING; otherwise those columns are updated from the proposed row.

copy_records

copy_records(
    table: str,
    columns: Sequence[str],
    records: Any,
    *,
    target: DatabaseTarget,
    timeout: float | None = None,
) -> ExecutionResult

Bulk-ingest records into table via PostgreSQL COPY (§19.2).

records is an iterable (async or sync) of row sequences matching columns. Fastest path for large ingests; PostgreSQL + psycopg only.