feat(rpc)!: add lightwalletd-compatible gRPC server implementing CompactTxStreamer

[arya2]

Motivation

Zcash light clients currently need a separate lightwalletd instance sitting between them and a full node. This PR lets Zebra serve light clients directly by implementing the lightwalletd CompactTxStreamer gRPC interface in zebra-rpc.

Close #11158

Solution

Adds a new zebra_rpc::lightwalletd module: a tonic gRPC server implementing all 18 CompactTxStreamer methods, enabled with a new rpc.lightwalletd_listen_addr config field (disabled by default).

• The service.proto / compact_formats.proto files from zcash/lightwalletd are vendored into zebra-rpc/proto/ and compiled by the existing build-script mechanism, with generated code checked into proto/__generated__/ so crates.io builds don't need protoc.
• Wallet-shaped methods (GetTreeState, GetSubtreeRoots, GetTaddressBalance, GetTaddressTxids, GetAddressUtxos, SendTransaction, GetLightdInfo) call the existing JSON-RPC RpcImpl methods in-process — the same methods lightwalletd calls on zcashd/Zebra over HTTP.
• Compact blocks (GetBlock, GetBlockRange, and the nullifier variants) and mempool streams (GetMempoolTx, GetMempoolStream) are built directly from the read-state and mempool services, following the existing indexer gRPC server's patterns. Compact-block tree sizes are fetched by the block's own hash so a concurrent reorg can't make ChainMetadata inconsistent with the block.
• Wire-format details were verified against the lightwalletd Go implementation, including two places where lightwalletd's behavior contradicts the proto comments: Exclude entries in GetMempoolTx are little-endian txid suffixes, and GetMempoolStream sends height: 0 for mempool transactions (Figure out what to do in librustzcash about the broken GetTransaction method of lightwalletd zcash/librustzcash#1484).
• Ping is disabled, matching production lightwalletd instances.

Tests

• New unit tests for BlockID conversion and Exclude list matching (byte order, ambiguous and empty entries).
• All existing zebra-rpc tests pass (cargo nextest run -p zebra-rpc --release: 113 passed).
• Manual end-to-end test on Regtest: started zebrad with lightwalletd_listen_addr set, mined 5 blocks via the generate RPC, and exercised 12 methods with a tonic CompactTxStreamerClient (GetLightdInfo, GetLatestBlock, GetBlock by height and by hash, GetBlockRange, GetTreeState, GetLatestTreeState, GetMempoolTx, GetTaddressBalance, GetTaddressTxids, GetAddressUtxos), verifying hashes, heights, coinbase balances, and tree states.
• cargo fmt --check and cargo clippy --workspace --all-targets are clean for the touched crates.

Specifications & References

• https://github.com/zcash/lightwalletd/blob/master/walletrpc/service.proto
• https://github.com/zcash/lightwalletd/blob/master/walletrpc/compact_formats.proto
• Figure out what to do in librustzcash about the broken GetTransaction method of lightwalletd zcash/librustzcash#1484 (RawTransaction height semantics)

Follow-up Work

• GetBlockRange issues 3 sequential state reads per block; pipelining them would matter for full-chain light-client sync throughput.
• pub use wire::* re-exports all generated proto types from zebra-rpc; a curated re-export would shrink the semver-checked public API surface.
• The indexer and lightwalletd gRPC servers share near-identical spawn code that could be deduplicated.
• Note: fixing the build script's descriptor-copy comparison (binary-safe fs::read) means checked-in *_descriptor.bin files now refresh from local protoc output, which regenerated indexer_descriptor.bin in this PR and may churn across contributor protoc versions.

AI Disclosure

• AI tools were used: Claude Code wrote the implementation, tests, and this PR description, with multi-agent review; the author directed and verified the work.

PR Checklist

• The PR title follows conventional commits format: type(scope): description
• The PR follows the contribution guidelines.
• This change was discussed in an issue or with the team beforehand.
• The solution is tested.
• The documentation and changelogs are up to date.

[Copilot]

Pull request overview

Adds an opt-in CompactTxStreamer gRPC server so Zebra can directly serve light clients.

Changes:

• Implements gRPC handlers using Zebra RPC, state, tip, and mempool services.
• Adds configuration, startup wiring, vendored protobufs, and generated bindings.
• Updates user and crate changelogs.

Assessment: The implementation targets an outdated lightwalletd schema and has unresolved correctness, DoS, backpressure, and test-coverage issues. The supplied metadata also lacks a link to the pre-discussed issue or maintainer acknowledgment.

Reviewed changes

Copilot reviewed 12 out of 15 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
CHANGELOG.md	Documents the new server.
zebra-rpc/CHANGELOG.md	Records the crate API addition.
zebra-rpc/build.rs	Builds or copies protobuf artifacts.
zebra-rpc/proto/compact_formats.proto	Defines compact block messages.
zebra-rpc/proto/service.proto	Defines CompactTxStreamer.
zebra-rpc/proto/__generated__/cash.z.wallet.sdk.rpc.rs	Generated tonic bindings.
zebra-rpc/src/config/rpc.rs	Adds the listen address setting.
zebra-rpc/src/lib.rs	Exposes the module.
zebra-rpc/src/lightwalletd.rs	Converts chain data to compact messages.
zebra-rpc/src/lightwalletd/methods.rs	Implements the gRPC methods.
zebra-rpc/src/lightwalletd/server.rs	Initializes the tonic server.
zebra-rpc/src/server/tests/vectors.rs	Updates configuration fixtures.
zebrad/src/commands/start.rs	Starts and monitors the server.

[oxarbitrage]

Summary of what's been pushed here since Friday.

Merged main, and worked through the review and Copilot threads — all 13 now resolved, CI green. Most of it is hardening: transport limits and HTTP/2 keepalives, bounded sends on every stream, address-count and exclude-list caps, and a status-code pass so client input isn't reported as a server fault.

Two behaviour changes worth calling out. A missing commitment tree now returns ABORTED rather than a zero tree size — previously a reorg between the block read and the tree read produced a CompactBlock claiming no notes existed, corrupting every note position derived from it. And completingBlockHash now goes out in display order, matching lightwalletd's GetSubtreeRoots.

Both have tests that fail without the fix, using a new in-process harness at lightwalletd/tests/vectors.rs — it binds a real port, starts the real server against mocked services, and connects a tonic client.

Two findings were split out rather than grown into this PR: #11238 (re-vendoring the protos for transparent vin/vout) and #11239 (per-request work caps, an optional service's failure taking down the node, and the mempool stream's timeout semantics).