Stream Queues¶
Consuming streams (offsets, offset stores, delivery semantics): see the Streams guide.
StreamConsumerConfig
dataclass
¶
How a route consumes a stream queue (subscriber(stream=...)).
Attributes:
| Name | Type | Description |
|---|---|---|
offset |
StreamOffset
|
where to start when offset_store holds nothing for this
consumer ( |
consumer_name |
str | None
|
the consumer's identity: the key its offset is stored
under (default: the queue name) and the |
offset_store |
OffsetStore | None
|
where the processed offset is committed. |
commit_every |
int
|
commit after this many newly processed messages. |
commit_interval |
float | None
|
also commit when this many seconds passed since the
last commit (checked when a message is settled; |
Source code in src/rabbitkit/core/streams.py
StreamOffset
dataclass
¶
Where a stream consumer starts when no stored offset exists.
Use the class methods, or :meth:parse::
StreamOffset.first()
StreamOffset.last()
StreamOffset.next()
StreamOffset.offset(42)
StreamOffset.timestamp(datetime(2026, 1, 1, tzinfo=UTC))
StreamOffset.parse("first") # also an int or a datetime
A naive datetime is taken as UTC.
Source code in src/rabbitkit/core/streams.py
62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 | |
Methods:¶
first() -> StreamOffset
classmethod
¶
last() -> StreamOffset
classmethod
¶
next() -> StreamOffset
classmethod
¶
offset(value: int) -> StreamOffset
classmethod
¶
timestamp(value: datetime) -> StreamOffset
classmethod
¶
Start from the first chunk published at or after value (naive = UTC).
Chunk granularity: a few messages published just before value may be delivered too.
Source code in src/rabbitkit/core/streams.py
parse(value: StreamOffset | str | int | datetime) -> StreamOffset
classmethod
¶
"first" / "last" / "next", an int offset, a datetime,
or a :class:StreamOffset (returned as is).
Source code in src/rabbitkit/core/streams.py
to_consume_arguments() -> dict[str, Any]
¶
The basic.consume arguments for this start position.
A timestamp is passed as a datetime: both pika and aio-pika encode
it as an AMQP timestamp, which RabbitMQ reads as a time. (An integer
would be read as a numeric offset.)
Source code in src/rabbitkit/core/streams.py
StreamOffsetType
¶
Bases: str, Enum
Where a stream consumer starts reading (RabbitMQ x-stream-offset).
- FIRST: the oldest message still retained by the stream.
- LAST: the last chunk written (the most recent messages).
- NEXT: only messages published after the consumer subscribes (the broker's default).
- OFFSET: a numeric offset.
- TIMESTAMP: the first chunk published at or after a point in time.
Source code in src/rabbitkit/core/types.py
OffsetStore
¶
Bases: Protocol
Where a stream consumer's processed offset is committed.
key identifies the consumer (StreamConsumerConfig.consumer_name,
else the queue name). save stores the last processed offset; the
consumer resumes at load(key) + 1.
rabbitkit ships only :class:InMemoryOffsetStore (offsets survive
reconnects, not restarts). For restarts, implement these two methods over
the storage your service already uses (its database, a key-value store, a
file), ideally in the same transaction as the handler's own writes.
Called from whichever thread settles messages (SyncBroker: the I/O or a
worker thread) or from a worker thread (AsyncBroker offloads every
store except :class:InMemoryOffsetStore), so implementations must be
thread-safe. save is only called with increasing offsets per key
within one process.
Source code in src/rabbitkit/core/streams.py
InMemoryOffsetStore
¶
Offsets kept in process memory: survive reconnects, not restarts.
The default when a stream route sets no offset_store.