Processor-related fixes#89
Draft
C-Achard wants to merge 29 commits into
Draft
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses two processor-related UX/plug-in discovery issues in DeepLabCut-live-GUI: (1) ensuring DLC/processor controls correctly re-enable after preview stop, and (2) improving processor class discovery so indirect dlclive.Processor subclasses can be surfaced in the GUI.
Changes:
- Adds a shared
discover_processor_classes()helper to finddlclive.Processorsubclasses (including indirect subclasses) and reuses it for both package and file-based processor loading. - Hooks
allow_processor_ctrl_checkboxchanges into UI update routines, and refreshes DLC control enabled-state on multi-camera preview start/stop. - Simplifies DLC control enabling logic so processor-related widgets re-enable based on inference state.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| dlclivegui/processors/processor_utils.py | Refactors processor discovery into reusable helpers and uses them for package/file scanning. |
| dlclivegui/gui/main_window.py | Updates signal wiring and control enable/disable logic to restore processor/DLC UI states after preview transitions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
b7c1b12 to
eaee775
Compare
790aeef to
52f9b92
Compare
Introduce `FrameTimestampMetadata` in `dlclivegui/utils/timestamps.py` to standardize optional backend/hardware timestamp data for captured frames. The dataclass captures source/backend fields, converted and raw timestamp values, conversion metadata, and backend extras, and adds helper methods to serialize source-level data, per-frame values, full dictionaries, and the configured default reported timestamp.
Adds end-to-end support for optional per-frame hardware timestamp metadata. Camera backends now return a `CapturedFrame` object (while preserving tuple unpacking), multi-camera signals and recording paths carry timestamp metadata, and `VideoRecorder` persists richer timestamp records. The timestamp JSON output is upgraded to schema v2 with backward-compatible software `timestamps` plus source metadata and per-frame hardware timestamp fields.
Refactors backend tests to use the new `read()` return payload (`CapturedFrame`) instead of tuple unpacking, including frame/timestamp access updates and minor unused-variable cleanup. Test fixtures were also aligned with timestamp metadata support by returning `CapturedFrame` in the fake backend and extending fake recorder/frame callback signatures to accept `timestamp_metadata`.
Updates the recording metadata schema in `video_recorder.py` by renaming `hardware_frame_timestamps` to `frame_timestamps`. This aligns timestamp data with a more general key name while preserving the same underlying values.
Expands test coverage for frame timestamp metadata end-to-end: Basler backend reads now validate hardware timestamp extraction, controller/recording manager tests assert metadata forwarding, and video recorder tests verify schema_version 2 sidecar output for both software-only and hardware-backed timestamps. Also adds focused unit tests for `FrameTimestampMetadata` source/frame field splitting and default-reported value behavior.
Improve Basler hardware timestamp robustness by treating support as best-effort, recording the tick-frequency source, falling back to an assumed 1 GHz clock when frequency is unavailable, and ignoring zero-value camera timestamps as missing data. The frame timestamp metadata now includes the frequency source in `extra`, and unused last-frame timestamp state was removed. Video recorder metadata also drops the legacy top-level `timestamps` field in favor of the structured `timestamp_sources` schema.
Updates `test_video_recorder.py` to stop asserting the top-level `timestamps` list in sidecar JSON fixtures. The tests now focus on schema v2 fields that remain authoritative (`frame_timestamps`, `start_time`, `end_time`, `duration_seconds`, and timestamp source metadata), aligning expectations with current sidecar output.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Disable all DLC and processor configuration widgets consistently while inference is active, including the processor-control checkbox. Refactor processor discovery into shared helpers that detect direct and indirect `dlclive.Processor` subclasses, standardize metadata extraction, and reuse the same fallback logic for package scans and file-based loading.
Expand processor class discovery to include re-exported classes by disabling module-only filtering in package/file scans. Also broaden subclass-check error handling to catch unexpected exceptions and log full context when discovery encounters problematic objects.
Create `dlclivegui/processors/__init__.py` to re-export `register_processor`, `BaseProcessorSocket`, and `PROCESSOR_REGISTRY` from `dlc_processor_socket`, making these APIs available via package-level imports.
Refactors `dlc_processor_socket.py` by removing the in-file example processors and `OneEuroFilter`, and adds them to a new `dlclivegui/processors/examples.py` module. This separates demonstration/experiment-specific logic from the core socket processor implementation, improving maintainability while preserving existing example processor behavior.
Refines `PLUGIN_SYSTEM.md` to reflect the current processor structure: it now points to `examples.py` for sample implementations and keeps `dlc_processor_socket.py` focused on the socket base class. The registration example was also updated to import `register_processor` and `PROCESSOR_REGISTRY` from `dlclivegui.processors` instead of redefining them inline.
Update processor package discovery to ignore `dlc_processor_socket` during namespace scanning, since it only provides the base class/registry and should not be listed as an available processor source. The package fallback scan now uses default class discovery behavior, and related outdated comments/docstring lines were cleaned up.
Change `register_processor` to log a warning instead of raising on duplicate `PROCESSOR_ID` keys, allowing later registrations to override earlier ones without import-time failures. Update subclass save tests to load processor classes from `dlclivegui.processors.examples` via a dedicated fixture, so the parametrized tests validate the concrete example processors against the correct module data path.
Updates `scan_processor_package` to use a more precise return type annotation (`dict[str, dict]`
Update processor imports to use `from dlclive.processor import Processor` in runtime code to avoid torch import side effects
Moves processor registration and discovery helpers out of `dlc_processor_socket.py` into a new `registry.py` module so registry access no longer depends on importing socket logic. `dlc_processor_socket.py` now imports the shared registry helpers and adds a safe fallback when `dlclive` is unavailable, reducing import-time failures in environments without that dependency. Package exports were updated to expose registry APIs from the new module.
Update the base processor test helper to better mirror the real dlclive package layout by mocking both `dlclive` and `dlclive.processor`, and add a no-op `process` method on the dummy `Processor`. This prevents import/behavior mismatches in tests that rely on the processor interface.
Update `Engine` to inherit from `str, Enum` so enum members behave like strings where needed. Also harden `from_model_type` by coercing non-string inputs (including enum-like values with `.value`) before lowercasing, and raise a clear `ValueError` when conversion is not possible.
Remember the processor folder across sessions and use it when initializing the main window. The folder is now saved when browsing, during refresh (after resolving a valid directory), and on close. Processor refresh messaging was updated to show whether processors came from the selected folder or the built-in package. Settings store gained processor-folder get/set helpers that validate and normalize paths, with safe fallback to defaults when paths are missing or invalid.
Enhance error reporting and handling for video recording. recording_manager now logs exception type, message, and frame shape/dtype when a write fails. VideoRecorder adds detailed messages for frame-size mismatches, queue retrieval errors, and encoding failures (including frame description, expected size, frames_written/frames_enqueued/dropped, and queue_size) and stops the recorder to avoid FFmpeg pipe errors. Introduced _describe_frame to summarize frames and _set_encode_error to centralize creation of a RuntimeError (preserving original exception as __cause__) and set _encode_error under the stats lock. Minor test file newline fix.
52f9b92 to
a800c08
Compare
Comment on lines
119
to
+123
| with self._timing.measure("Single.read"): | ||
| frame, timestamp = self._backend.read() | ||
| captured = self._backend.read() | ||
| frame = captured.frame | ||
| timestamp = captured.software_timestamp | ||
| timestamp_metadata = captured.timestamp_metadata |
Comment on lines
+17
to
+20
| try: | ||
| from dlclive.processor import Processor # type: ignore | ||
| except ImportError: | ||
| Processor = object # Fallback for type checking if dlclive is not installed |
Comment on lines
+20
to
+23
| def _processor_base_class(): | ||
| from dlclive.processor import Processor | ||
|
|
||
| return Processor |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #87 and closes #88