It always ends on a block boundary. A block’s rows are never split across two responses, so
a response can overshoot
maxObjects or 8 MB by up to one block’s rows.
The loop
Every response carriesnext_block: the fromBlock for the next page. Keep going until
next_block passes the end of your range.
The client libraries do this for you with paginate:
Writing the loop yourself
query is the reader from Reading Arrow; with /query/json the
same value is the next_block field of the body.
A real run, all transfers of three tokens over 1,500 Ethereum blocks with maxObjects: 20000:
Rules worth knowing
- When everything fit,
next_blockis simplyto_block + 1. - Past the head, a request returns no rows and
next_blockequals yourfromBlock. A client waiting for the next block just repeats the request; see Following the head. - An open-ended range (no
toBlock) stops at the head. Loop untilnext_blockpasses thearchive_heightyou saw, or keep polling. - There is no “truncated” flag.
next_blockalone tells you whether there’s more: compare it with the end you want. - A small
maxObjectscan still return many rows.maxObjects: 1on a busy block returns that whole block (hundreds of rows), because blocks are never split.
Choosing maxObjects
- Leave the default (100,000) for backfills. Bigger pages mean fewer requests, which matters against your requests-per-second limit.
- Use smaller pages if you process each page slowly. A response holds one of the chain’s query slots until you’ve read it all, and a client that stops reading is cut off after 60 seconds. Read the response promptly, then do your slow work.
/query/jsoncaps a page at 100,000 rows too, but JSON is bigger, so smaller pages are friendlier to your memory.
Pages and cost
Paging doesn’t change the cost: you pay for the rows you receive, wherever the page boundaries fall. More pages do cost more requests, and each request is billed at least 10 KB. For a narrow filter over a long range (few rows, many 50,000-block pages) that minimum is most of the cost. It is still small: 1,000 pages is 10 MB, half a cent on pay-as-you-go.Speeding up repeated filters
A backfill sends the same filter on every page. Two features make that cheaper:- Prepared queries: send the filter once, then only its id and the range.
- Sets: upload a long address list once, refer to it by id.