Skip to content

Processor-related fixes#89

Draft
C-Achard wants to merge 29 commits into
cy/improve-encoding-speedfrom
cy/processor-fixes
Draft

Processor-related fixes#89
C-Achard wants to merge 29 commits into
cy/improve-encoding-speedfrom
cy/processor-fixes

Conversation

@C-Achard

@C-Achard C-Achard commented Jun 30, 2026

Copy link
Copy Markdown
Contributor
  • Fix processor field not re-enabling after stopping preview
  • Fix Processor child classes not properly showing up in the UI
  • Move examples out of base class file
  • Ensure base class is not loaded in examples
  • Ensure documentation is up to date

Closes #87 and closes #88

@C-Achard C-Achard requested a review from Copilot June 30, 2026 14:43
@C-Achard C-Achard self-assigned this Jun 30, 2026
@C-Achard C-Achard added enhancement New feature or request gui Related to the GUI itself : windows and fields bugs, UI, UX, ... labels Jun 30, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses two processor-related UX/plug-in discovery issues in DeepLabCut-live-GUI: (1) ensuring DLC/processor controls correctly re-enable after preview stop, and (2) improving processor class discovery so indirect dlclive.Processor subclasses can be surfaced in the GUI.

Changes:

  • Adds a shared discover_processor_classes() helper to find dlclive.Processor subclasses (including indirect subclasses) and reuses it for both package and file-based processor loading.
  • Hooks allow_processor_ctrl_checkbox changes into UI update routines, and refreshes DLC control enabled-state on multi-camera preview start/stop.
  • Simplifies DLC control enabling logic so processor-related widgets re-enable based on inference state.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
dlclivegui/processors/processor_utils.py Refactors processor discovery into reusable helpers and uses them for package/file scanning.
dlclivegui/gui/main_window.py Updates signal wiring and control enable/disable logic to restore processor/DLC UI states after preview transitions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dlclivegui/processors/processor_utils.py
Comment thread dlclivegui/processors/processor_utils.py
Comment thread dlclivegui/processors/processor_utils.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Comment thread dlclivegui/processors/processor_utils.py Outdated
Comment thread dlclivegui/processors/processor_utils.py
Comment thread dlclivegui/processors/examples.py Outdated
Comment thread dlclivegui/processors/examples.py Outdated
Comment thread dlclivegui/processors/examples.py Outdated
@C-Achard C-Achard force-pushed the cy/processor-fixes branch from b7c1b12 to eaee775 Compare July 1, 2026 14:21
@C-Achard C-Achard changed the base branch from master to cy/improve-encoding-speed July 1, 2026 14:29
@C-Achard C-Achard force-pushed the cy/processor-fixes branch from 790aeef to 52f9b92 Compare July 2, 2026 14:29
C-Achard and others added 18 commits July 2, 2026 16:35
Introduce `FrameTimestampMetadata` in `dlclivegui/utils/timestamps.py` to standardize optional backend/hardware timestamp data for captured frames. The dataclass captures source/backend fields, converted and raw timestamp values, conversion metadata, and backend extras, and adds helper methods to serialize source-level data, per-frame values, full dictionaries, and the configured default reported timestamp.
Adds end-to-end support for optional per-frame hardware timestamp metadata. Camera backends now return a `CapturedFrame` object (while preserving tuple unpacking), multi-camera signals and recording paths carry timestamp metadata, and `VideoRecorder` persists richer timestamp records. The timestamp JSON output is upgraded to schema v2 with backward-compatible software `timestamps` plus source metadata and per-frame hardware timestamp fields.
Refactors backend tests to use the new `read()` return payload (`CapturedFrame`) instead of tuple unpacking, including frame/timestamp access updates and minor unused-variable cleanup. Test fixtures were also aligned with timestamp metadata support by returning `CapturedFrame` in the fake backend and extending fake recorder/frame callback signatures to accept `timestamp_metadata`.
Updates the recording metadata schema in `video_recorder.py` by renaming `hardware_frame_timestamps` to `frame_timestamps`. This aligns timestamp data with a more general key name while preserving the same underlying values.
Expands test coverage for frame timestamp metadata end-to-end: Basler backend reads now validate hardware timestamp extraction, controller/recording manager tests assert metadata forwarding, and video recorder tests verify schema_version 2 sidecar output for both software-only and hardware-backed timestamps. Also adds focused unit tests for `FrameTimestampMetadata` source/frame field splitting and default-reported value behavior.
Improve Basler hardware timestamp robustness by treating support as best-effort, recording the tick-frequency source, falling back to an assumed 1 GHz clock when frequency is unavailable, and ignoring zero-value camera timestamps as missing data. The frame timestamp metadata now includes the frequency source in `extra`, and unused last-frame timestamp state was removed. Video recorder metadata also drops the legacy top-level `timestamps` field in favor of the structured `timestamp_sources` schema.
Updates `test_video_recorder.py` to stop asserting the top-level `timestamps` list in sidecar JSON fixtures. The tests now focus on schema v2 fields that remain authoritative (`frame_timestamps`, `start_time`, `end_time`, `duration_seconds`, and timestamp source metadata), aligning expectations with current sidecar output.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Disable all DLC and processor configuration widgets consistently while inference is active, including the processor-control checkbox. Refactor processor discovery into shared helpers that detect direct and indirect `dlclive.Processor` subclasses, standardize metadata extraction, and reuse the same fallback logic for package scans and file-based loading.
Expand processor class discovery to include re-exported classes by disabling module-only filtering in package/file scans. Also broaden subclass-check error handling to catch unexpected exceptions and log full context when discovery encounters problematic objects.
Create `dlclivegui/processors/__init__.py` to re-export `register_processor`, `BaseProcessorSocket`, and `PROCESSOR_REGISTRY` from `dlc_processor_socket`, making these APIs available via package-level imports.
Refactors `dlc_processor_socket.py` by removing the in-file example processors and `OneEuroFilter`, and adds them to a new `dlclivegui/processors/examples.py` module. This separates demonstration/experiment-specific logic from the core socket processor implementation, improving maintainability while preserving existing example processor behavior.
Refines `PLUGIN_SYSTEM.md` to reflect the current processor structure: it now points to `examples.py` for sample implementations and keeps `dlc_processor_socket.py` focused on the socket base class. The registration example was also updated to import `register_processor` and `PROCESSOR_REGISTRY` from `dlclivegui.processors` instead of redefining them inline.
Update processor package discovery to ignore `dlc_processor_socket` during namespace scanning, since it only provides the base class/registry and should not be listed as an available processor source. The package fallback scan now uses default class discovery behavior, and related outdated comments/docstring lines were cleaned up.
C-Achard added 11 commits July 2, 2026 16:36
Change `register_processor` to log a warning instead of raising on duplicate `PROCESSOR_ID` keys, allowing later registrations to override earlier ones without import-time failures. Update subclass save tests to load processor classes from `dlclivegui.processors.examples` via a dedicated fixture, so the parametrized tests validate the concrete example processors against the correct module data path.
Updates `scan_processor_package` to use a more precise return type annotation (`dict[str, dict]`
Update processor imports to use `from dlclive.processor import Processor` in runtime code to avoid torch import side effects
Moves processor registration and discovery helpers out of `dlc_processor_socket.py` into a new `registry.py` module so registry access no longer depends on importing socket logic. `dlc_processor_socket.py` now imports the shared registry helpers and adds a safe fallback when `dlclive` is unavailable, reducing import-time failures in environments without that dependency. Package exports were updated to expose registry APIs from the new module.
Update the base processor test helper to better mirror the real dlclive package layout by mocking both `dlclive` and `dlclive.processor`, and add a no-op `process` method on the dummy `Processor`. This prevents import/behavior mismatches in tests that rely on the processor interface.
Update `Engine` to inherit from `str, Enum` so enum members behave like strings where needed. Also harden `from_model_type` by coercing non-string inputs (including enum-like values with `.value`) before lowercasing, and raise a clear `ValueError` when conversion is not possible.
Remember the processor folder across sessions and use it when initializing the main window. The folder is now saved when browsing, during refresh (after resolving a valid directory), and on close. Processor refresh messaging was updated to show whether processors came from the selected folder or the built-in package. Settings store gained processor-folder get/set helpers that validate and normalize paths, with safe fallback to defaults when paths are missing or invalid.
Enhance error reporting and handling for video recording. recording_manager now logs exception type, message, and frame shape/dtype when a write fails. VideoRecorder adds detailed messages for frame-size mismatches, queue retrieval errors, and encoding failures (including frame description, expected size, frames_written/frames_enqueued/dropped, and queue_size) and stops the recorder to avoid FFmpeg pipe errors. Introduced _describe_frame to summarize frames and _set_encode_error to centralize creation of a RuntimeError (preserving original exception as __cause__) and set _encode_error under the stats lock. Minor test file newline fix.
@C-Achard C-Achard force-pushed the cy/processor-fixes branch from 52f9b92 to a800c08 Compare July 2, 2026 14:36
@C-Achard C-Achard changed the base branch from cy/improve-encoding-speed to cy/hardware-timestamps July 2, 2026 14:37
@C-Achard C-Achard changed the base branch from cy/hardware-timestamps to cy/improve-encoding-speed July 2, 2026 14:37
@C-Achard C-Achard requested a review from Copilot July 2, 2026 15:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 3 comments.

Comment on lines 119 to +123
with self._timing.measure("Single.read"):
frame, timestamp = self._backend.read()
captured = self._backend.read()
frame = captured.frame
timestamp = captured.software_timestamp
timestamp_metadata = captured.timestamp_metadata
Comment on lines +17 to +20
try:
from dlclive.processor import Processor # type: ignore
except ImportError:
Processor = object # Fallback for type checking if dlclive is not installed
Comment on lines +20 to +23
def _processor_base_class():
from dlclive.processor import Processor

return Processor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request gui Related to the GUI itself : windows and fields bugs, UI, UX, ...

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow Processor child classes to be loaded in the GUI Re-enable processor field when disabling preview

2 participants