Skip to content

perf(consensus): batch block transaction UTXO lookups - #11186

Draft
upbqdn wants to merge 2 commits into
mainfrom
11185-batch-block-utxo-lookups
Draft

perf(consensus): batch block transaction UTXO lookups#11186
upbqdn wants to merge 2 commits into
mainfrom
11185-batch-block-utxo-lookups

Conversation

@upbqdn

@upbqdn upbqdn commented Aug 5, 2026

Copy link
Copy Markdown
Member

Motivation

Closes #11185.

Solution

block_spent_utxos issued one zebra_state::Request::AwaitUtxo per transparent input and awaited
each before starting the next, so a transaction with many inputs serialized that many round trips
through the state service.

Adds zebra_state::Request::AnyChainUtxos, which takes a list of outpoints and returns only the ones
already in the queued blocks, any non-finalized chain, or the finalized state. Unlike AwaitUtxo it
never waits and never registers a pending UTXO request, so the block verifier uses it for the bulk of
its lookups and falls back to AwaitUtxo only for outpoints that have not reached the state yet.

Lookups go out in chunks of 64 and stop at the first chunk that is not fully present. That bound is
the reason for chunking rather than sending one request: without it, a transaction whose inputs are
all unknown would make the state look up every input before the sequential fallback failed on the
first, which is more work than the one lookup the old loop did before its timeout. With it, at most
one chunk is speculative.

To be precise about what that bound is and is not: it is additive. A transaction that references
outputs which really are in the state is still looked up in full, exactly as before. The worst case
is old cost plus one chunk, never a multiple of the input count.

Tests

Three new tests in zebra-consensus/src/transaction/tests.rs, each built with a multi-input
transaction and a MockService state, so they assert the request sequence rather than just the
outcome:

  • block_utxo_lookups_use_one_bulk_request — one AnyChainUtxos carrying every outpoint in input
    order, then no further state requests.
  • block_utxo_lookups_await_only_the_missing_outpoints — one outpoint withheld from the bulk
    response is awaited with AwaitUtxo, and the found ones are not.
  • block_utxo_lookups_stop_at_the_first_chunk_with_misses — with 72 inputs and an empty bulk
    response, only the first 64 are ever requested.

Each was checked against the mutation it is meant to catch, and fails without the fix: reversing the
expected outpoint order, withholding a second outpoint, and removing the early break respectively.

Specifications & References

Follow-up Work

PR Checklist

upbqdn added 2 commits August 5, 2026 02:07
Request::AwaitUtxo takes one outpoint and registers a pending request for it,
so a caller with many outpoints needs one round trip through the state service
per outpoint.

Add Request::AnyChainUtxos, which takes a list and returns only the outpoints
already present in the queued blocks, any non-finalized chain, or the finalized
state. It never waits and never registers a pending request, so callers keep
using AwaitUtxo for the outpoints it did not find.

ReadRequest::AnyChainUtxo becomes AnyChainUtxos: its only caller was AwaitUtxo,
which now passes a single-element list.
block_spent_utxos issued one Request::AwaitUtxo per transparent input and
awaited each before starting the next, so a transaction with many inputs
serialized that many round trips through the state service, and the block
verifier could not finish until every transaction had done so.

Look the outpoints up with Request::AnyChainUtxos in chunks of
UTXO_LOOKUP_CHUNK_SIZE instead, falling back to AwaitUtxo for outpoints that
have not reached the state yet.

The lookups stop at the first chunk that is not fully present, so batching
cannot make a transaction cost the state more lookups than the sequential
version did: at most one chunk is speculative. This is an additive bound, not
independence from the input count, since a transaction referencing outputs that
really are in the state is still looked up in full.

Closes #11185.
@v12-auditor

v12-auditor Bot commented Aug 5, 2026

Copy link
Copy Markdown

Note

Complete: Audit complete. V12 did not find any issues that need review.

Open the full results here.

Analyzed six files, diff 5098bdc...6a3df2b.

@upbqdn upbqdn added C-enhancement Category: This is an improvement C-breaking Category: A breaking change for users I-slow Problems with performance or responsiveness A-consensus Area: Consensus rule updates A-state Area: State / database changes labels Aug 5, 2026
@upbqdn upbqdn self-assigned this Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-consensus Area: Consensus rule updates A-state Area: State / database changes C-breaking Category: A breaking change for users C-enhancement Category: This is an improvement I-slow Problems with performance or responsiveness

Projects

None yet

Development

Successfully merging this pull request may close these issues.

zebra-consensus: batch the per-input UTXO lookups during block transaction verification

1 participant