> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bloklake.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Pagination

> Read any range, however large, with the next_block loop.

A response covers part of your range at most. It ends at the **first** of:

| Limit | Value |
| - | - |
| Objects (blocks + transactions + logs) | `maxObjects`, default and maximum 100,000 |
| Data | about 8 MB of rows |
| Block span | 50,000 blocks |
| Your `toBlock` (or the head) | — |

It always ends on a **block boundary**. A block's rows are never split across two responses, so
a response can overshoot `maxObjects` or 8 MB by up to one block's rows.

## The loop

Every response carries `next_block`: the `fromBlock` for the next page. Keep going until
`next_block` passes the end of your range.

The [client libraries](https://docs.bloklake.com/clients.md) do this for you with `paginate`:

```typescript TypeScript theme={null}
import { Bloklake } from "@bloklake/client";

const bl = new Bloklake({ chain: "eth" });
const TRANSFER = "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef";

let total = 0;
for await (const page of bl.paginate({
  fromBlock: 19426000,
  toBlock: 19427499,
  logs: [{ address: ["0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48"], topic0: [TRANSFER] }],
  fields: { log: ["blockNumber", "logIndex", "topic1", "topic2", "data"] },
})) {
  total += page.logs.length; // store(page.logs)
  console.log(`blocks ${page.meta.fromBlock}..${page.meta.toBlock}: ${page.logs.length} logs`);
}
console.log(total, "transfers");
```

```python Python theme={null}
from bloklake import Bloklake

bl = Bloklake(chain="eth")
TRANSFER = "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"

total = 0
for page in bl.paginate({
    "from_block": 19426000,
    "to_block": 19427499,
    "logs": [{"address": ["0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48"], "topic0": [TRANSFER]}],
    "fields": {"log": ["block_number", "log_index", "topic1", "topic2", "data"]},
}):
    total += len(page.logs)  # store(page.logs)
    print(f"blocks {page.meta.from_block}..{page.meta.to_block}: {len(page.logs)} logs")
print(total, "transfers")
```

```rust Rust theme={null}
use bloklake::{Client, LogField, LogFilter, Query};

const TRANSFER: &str = "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef";

#[tokio::main]
async fn main() -> bloklake::Result<()> {
    let client = Client::builder().chain("eth").build()?;
    let query = Query::new()
        .from_block(19426000)
        .to_block(19427499)
        .log(LogFilter::new().address(["0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48"]).topic0([TRANSFER]))
        .log_fields([LogField::BlockNumber, LogField::LogIndex, LogField::Topic1, LogField::Topic2, LogField::Data]);

    let mut total = 0;
    let mut pages = client.paginate(&query);
    while let Some(page) = pages.next().await? {
        total += page.logs.len(); // store(&page.logs)
        let m = page.meta.unwrap();
        println!("blocks {}..{}: {} logs", m.from_block, m.to_block, page.logs.len());
    }
    println!("{total} transfers");
    Ok(())
}
```

### Writing the loop yourself

```python Python theme={null}
def read_range(body: dict, start: int, end: int):
    frm = start
    while frm <= end:
        tables, meta = query({**body, "fromBlock": frm, "toBlock": end})
        yield tables
        frm = int(meta["bl.next_block"])
```

```typescript TypeScript theme={null}
async function* readRange(body: object, start: number, end: number) {
  let from = start;
  while (from <= end) {
    const { tables, meta } = await query({ ...body, fromBlock: from, toBlock: end });
    yield tables;
    from = Number(meta.get("bl.next_block"));
  }
}
```

```bash JSON + jq theme={null}
from=19426000; end=19427499
while [ "$from" -le "$end" ]; do
  page=$(curl -s https://api.bloklake.com/v1/chains/eth/query/json \
    -H "Authorization: Bearer $BLOKLAKE_API_KEY" -H 'content-type: application/json' \
    -d "{\"fromBlock\": $from, \"toBlock\": $end, \"logs\": [{\"address\": [\"0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48\"]}], \"fields\": {\"log\": {\"blockNumber\": true}}}")
  echo "$page" | jq -c '.logs[]'
  from=$(echo "$page" | jq .next_block)
done
```

`query` is the reader from [Reading Arrow](https://docs.bloklake.com/responses/reading-arrow.md); with `/query/json` the
same value is the `next_block` field of the body.

A real run, all transfers of three tokens over 1,500 Ethereum blocks with `maxObjects: 20000`:

```text theme={null}
page 1: blocks 19426000..19426298  logs 20004
page 2: blocks 19426299..19426581  logs 20023
page 3: blocks 19426582..19426860  logs 20087
page 4: blocks 19426861..19427114  logs 20015
page 5: blocks 19427115..19427384  logs 20080
page 6: blocks 19427385..19427499  logs  8921
```

Each page ends where its object count first reached 20,000, at the end of that block.

## Rules worth knowing

* **When everything fit**, `next_block` is simply `to_block + 1`.
* **Past the head**, a request returns no rows and `next_block` equals your `fromBlock`. A
  client waiting for the next block just repeats the request; see
  [Following the head](https://docs.bloklake.com/guides/follow-the-head.md).
* **An open-ended range** (no `toBlock`) stops at the head. Loop until `next_block` passes the
  `archive_height` you saw, or keep polling.
* **There is no "truncated" flag.** `next_block` alone tells you whether there's more: compare
  it with the end you want.
* **A small `maxObjects` can still return many rows.** `maxObjects: 1` on a busy block returns
  that whole block (hundreds of rows), because blocks are never split.

## Choosing `maxObjects`

* **Leave the default** (100,000) for backfills. Bigger pages mean fewer requests, which matters
  against your [requests-per-second limit](https://docs.bloklake.com/production/rate-limits.md).
* **Use smaller pages** if you process each page slowly. A response holds one of the chain's
  query slots until you've read it all, and a client that stops reading is
  [cut off after 60 seconds](https://docs.bloklake.com/production/errors-and-retries.md#slow-and-idle-clients). Read
  the response promptly, then do your slow work.
* **`/query/json`** caps a page at 100,000 rows too, but JSON is bigger, so smaller pages
  are friendlier to your memory.

## Pages and cost

Paging doesn't change the cost: you pay for the rows you receive, wherever the page boundaries
fall. More pages do cost more requests, and each request is
[billed at least 10 KB](https://docs.bloklake.com/billing/how-billing-works.md#minimum-per-request). For a narrow filter
over a long range (few rows, many 50,000-block pages) that minimum is most of the cost. It is
still small: 1,000 pages is 10 MB, half a cent on pay-as-you-go.

## Speeding up repeated filters

A backfill sends the same filter on every page. Two features make that cheaper:

* [Prepared queries](https://docs.bloklake.com/querying/prepared-queries.md): send the filter once, then only its id and
  the range.
* [Sets](https://docs.bloklake.com/querying/sets.md): upload a long address list once, refer to it by id.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.