Rework#9
Merged
Merged
Conversation
…ng and data synchronization logic
…cursor management and enhance block gap handling logic in indexer-actors
…header storage and retrieval methods
…ndary indexing for compact headers, integrate into block processing logic
…trieval, and prefix search for sender-based contextual messages, and integrate key struct enhancements for flexible queries
…for sender/receiver-based payment data and transaction-to-payment mappings
…sent or default, refine error handling, and add insertion safeguard in contextual_message
…xer-db: implement storage, retrieval, and processing logic for block-to-tx mappings and tx acceptance data
…transaction acceptance retrieval logic with prefix search and validation
… suffix handling, update methods for safer and more flexible transaction acceptance processing
…andshakes: implement sender/receiver key insertion, tx acceptance updates, and operational handling logic
…ndling, message processing, and synchronization logic for virtual chain notifications and blocks
… message processing logic, and introduce compact header integration for streamlined data management
…rt for contextual messages, payments, and handshakes, update partition logic, and refine processing methods for better flexibility and data organization
…, and key management for resolving pending sender transactions
…for enhanced state handling, update message processing logic, and integrate real-time queue processing
…kup and update acceptance logic, add LookupOutput enum for result handling, and refine VCC addition to track accepted transactions
…ge state using `StateShared`, enhance transaction tracking and DAA score processing during block removal
…on handling, integrate request processing with VCC notifications, and enhance queue management logic
…ution handling, add contextual message, payment, and handshake partition updates, and refine sender address processing logic
…SenderResolution handling, implement batch processing of pending resolutions, and add KnownLayout support for PendingResolutionKey
…odel, database partitioning, and real-time synchronization layers
IzioDev
reviewed
Aug 16, 2025
Comment on lines
+31
to
+33
| pub fn insert_wtx(&self, wtx: &mut WriteTransaction, key: &PaymentKeyBySender) { | ||
| wtx.insert(&self.0, key.as_bytes(), []); | ||
| } |
Contributor
There was a problem hiding this comment.
I see a similar operation on ContextualMessageBySenderPartition, it is named insert and does not take key as argument, but direct underlying data, so insert is responsible for the key building (and validation).
Would you consider db/messages be the validator for keys or should that concern be reserved to an upper calling layer, or even should we allow mixed signatures for the sake of flexibility?
add periodic processor to prune "expired" data and force metadata compaction add metrics
…ementation Add bon::Builder derive to BlockProcessor, VirtualProcessor, and PeriodicProcessor Complete main.rs actor initialization with proper builder patterns Add Ticker component for periodic maintenance task scheduling Update channel types from flume to workflow_core for consistency Refactor API layer with updated import paths and data access patterns Add serde_with support for hex serialization in metrics
use FSTR for block metrics
- Add extensive logging throughout indexer-actors crate with ToHex utilities - Fix data source shutdown channel from oneshot to mpsc for proper termination - Add debug logging to track shutdown sequence and identify hanging issues - Improve channel handling in data source with proper receiver access - Add metrics improvements and error handling enhancements - Update imports and dependencies for better organization - Add ticker with proper shutdown handling and logging
biryukovmaxim
marked this pull request as ready for review
August 17, 2025 05:20
…s to mpsc - Refactor shutdown system with `Shutdown` struct for centralized control. - Replace `oneshot` channels with `mpsc` for proper termination handling. - Add error context to improve diagnostics in `virtual_chain_syncer` and `data_source`. - Ensure consistent and robust error logging during shutdown and key async operations.
- Add `processed_time_or_warn` to track processing intervals. - Emit warning log if `VirtualChainChangedNotification` is not processed within 60 seconds. - Update timestamps after handling delayed processing. - Improve error handling and logging for missing blocks during removal.
- Update `handshakes_by_receiver` and `payments_by_receiver` to `uniq_handshakes_by_receiver` and `uniq_payments_by_receiver` for accuracy. - Adjust associated methods, structures, and format output for consistency.
- Comment out rayon usage and remove from dependencies. - Replace `par_iter` with sequential iteration for processed block collection. - Add trace and debug level logs to enhance clarity and debugging. - Refactor ticker shutdown handling for robustness and better error reporting. - Add improved logging to VirtualChainProcessor and VirtualChainSyncer processes.
…vant comments - Change `block.header.blue_work` serialization from big-endian to little-endian in `block_processor.rs`. - Update comment in `metadata.rs` to clarify BE (big-endian) byte representation.
… value updates - Replace manual value assembly with `fetch_update` for more concise and atomic updates. - Enhance handling of existing and new entries to improve performance and maintainability.
- Update warning messages to specify whether the delay involves the syncer or real-time VCC for improved logging clarity.
…dling - Simplify `VirtualProcessor` by removing the unused `unsynced_capacity` field. - Replace references to `unsynced_capacity` with `synced_capacity` for consistency. - Adjust default capacity of `synced_capacity` to 3,000,000.
- Introduce `TxIdToContextualMessagePartition` for mapping transaction IDs to sealed messages. - Enhance metrics to track contextual message counts with precision. - Replace legacy methods for managing contextual messages with more efficient implementations. - Refactor database operations for contextual messages to decouple sender and transaction ID mappings.
# Conflicts: # Cargo.toml # indexer-lib/examples/historical_block_sync.rs # indexer-lib/src/block_processor.rs # indexer-lib/src/database.rs # indexer-lib/src/database/messages/contextual_messages.rs # indexer-lib/src/database/messages/mod.rs # indexer-lib/src/database/processing/acceptance.rs # indexer-lib/src/database/processing/pending_sender_resolution.rs # indexer-lib/src/database/processing/unknown_daa_scores.rs # indexer-lib/src/database/resolution_keys.rs # indexer-lib/src/periodic_processor.rs # indexer-lib/src/virtual_chain_processor.rs # indexer/Cargo.toml # indexer/src/config.rs # indexer/src/main.rs_stash` logic.
* protocol: handshake v2 handling * review: fix naming * review: fix versioning and enrich logging --------- Co-authored-by: IzioDev <9900846+IzioDev@users.noreply.github.com>
* protocol: handshake v2 handling * review: fix naming * review: fix versioning and enrich logging --------- Co-authored-by: IzioDev <9900846+IzioDev@users.noreply.github.com>
# Conflicts: # protocol/src/operation.rs
Rework+self stash
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.