Paginate with the _iter Methods

Every read that can return more than one page — head, query, search, about, refs, collection members, repository and action listings — answers one page at a time: items plus a next_cursor. A next_cursor that is not None means there is more. A result that fits in one page has no cursor.

Each paged method has an _iter twin that does the cursor handling for you. Reach for it first.

Walk every page

from warmhub import WarmHubClient

with WarmHubClient.from_env() as client:
    sensors = client.repository("acme/sensors")

    for item in sensors.things.head_iter(shape="Reading", limit=10):
        print(item.wref, item.data)

limit= on an iterator is the page size, the number of rows per request. It is not a cap on how many rows you get back: the iterator still walks to the end. Over 40 matching readings, the loop above makes 4 requests.

The iterator also guards against a server that repeats a cursor, which would spin a hand-written loop forever. It raises instead.

Stop early

The iterators are lazy. They fetch the next page only when the loop asks for a row beyond the current one, so breaking out of the loop stops the paging:

from itertools import islice

first_five = list(islice(sensors.things.head_iter(shape="Reading", limit=10), 5))

That reads one page — one request — however many rows match.

Collect everything, with a ceiling

When you want a list rather than a loop, use the _all spelling: head_all, query_all, about_all, refs_all, and members_all. Pass max_items= to say how large the result is allowed to be:

vessels = client.repository("acme/fleet").things.head_all(shape="Vessel", max_items=500)

If more than max_items rows turn up, the call raises WarmHubError with code VALIDATION_ERROR rather than returning a list larger than you planned for.

Methods on client versus the repository handle

client.repository("org/repo").things carries head_iter, query_iter, head_all, and query_all. The other iterators live on the client namespaces, which take the organization and repository names explicitly:

hits = client.thing.search_iter("acme", "sensors", "bay", shape="Reading", limit=5)
claims = client.thing.about_iter("acme", "catalog", "bolt-m8", limit=25)
members = client.collection.members_all("acme", "fleet", "harbour-fleet")

Iterator	Namespace
head_iter, query_iter	client.thing, and the repository handle’s things
search_iter, about_iter, refs_iter, history_iter	client.thing
members_iter	client.collection
history_iter	client.shape
list_iter, list_page_iter, search_iter, explore_iter	client.repo
list_iter, search_iter	client.component
list_runs_iter, live_feed_iter	client.action

The Python SDK reference lists each method’s arguments.

Page by hand, and resume

When you need to stop and pick up later — across processes, or after a failure — keep the cursor. Read one page with the plain method and pass its next_cursor back as cursor=:

page = sensors.things.head(shape="Reading", limit=100)
handle(page.items)

if page.next_cursor is not None:
    page = sensors.things.head(shape="Reading", limit=100, cursor=page.next_cursor)

The iterators accept cursor= too, and start from it:

for item in sensors.things.head_iter(shape="Reading", limit=100, cursor=saved_cursor):
    handle_one(item)

Keep the other arguments the same when you resume. A cursor belongs to the read that produced it.

Async

On AsyncWarmHubClient, the _iter methods return async iterators. Use async for:

from warmhub import AsyncWarmHubClient

async with AsyncWarmHubClient.from_env() as client:
    sensors = client.repository("acme/sensors")
    async for item in sensors.things.head_iter(shape="Reading", limit=100):
        print(item.wref)

Search and glob reads are eventually consistent

search and glob match= reads are served from a separate search index that is updated asynchronously after each write. A search issued straight after a write can correctly return nothing. Before relying on a search result for data you just wrote, poll a glob count with a deadline:

import time

def await_indexed(repo, pattern: str, at_least: int, timeout_s: float = 60.0) -> None:
    deadline = time.monotonic() + timeout_s
    while int(repo.things.count(kind="thing", match=pattern).count) < at_least:
        if time.monotonic() >= deadline:
            raise TimeoutError(f"{pattern!r} did not reach {at_least} in {timeout_s}s")
        time.sleep(1.0)

await_indexed(sensors, "probe-*", 40)

A glob matches record names, so use a pattern over the names you wrote — the readings here are named probe-000 to probe-039 — not over their shape. The pattern ** matches everything and is answered without the search index, so a wait built on it returns immediately and proves nothing.

Anonymous callers are limited to 25 rows per page and two pages per walk; see Pagination.