Add hardware timestamp reporting#86
Draft
C-Achard wants to merge 12 commits into
Draft
Conversation
Introduce `FrameTimestampMetadata` in `dlclivegui/utils/timestamps.py` to standardize optional backend/hardware timestamp data for captured frames. The dataclass captures source/backend fields, converted and raw timestamp values, conversion metadata, and backend extras, and adds helper methods to serialize source-level data, per-frame values, full dictionaries, and the configured default reported timestamp.
Adds end-to-end support for optional per-frame hardware timestamp metadata. Camera backends now return a `CapturedFrame` object (while preserving tuple unpacking), multi-camera signals and recording paths carry timestamp metadata, and `VideoRecorder` persists richer timestamp records. The timestamp JSON output is upgraded to schema v2 with backward-compatible software `timestamps` plus source metadata and per-frame hardware timestamp fields.
Refactors backend tests to use the new `read()` return payload (`CapturedFrame`) instead of tuple unpacking, including frame/timestamp access updates and minor unused-variable cleanup. Test fixtures were also aligned with timestamp metadata support by returning `CapturedFrame` in the fake backend and extending fake recorder/frame callback signatures to accept `timestamp_metadata`.
Updates the recording metadata schema in `video_recorder.py` by renaming `hardware_frame_timestamps` to `frame_timestamps`. This aligns timestamp data with a more general key name while preserving the same underlying values.
Expands test coverage for frame timestamp metadata end-to-end: Basler backend reads now validate hardware timestamp extraction, controller/recording manager tests assert metadata forwarding, and video recorder tests verify schema_version 2 sidecar output for both software-only and hardware-backed timestamps. Also adds focused unit tests for `FrameTimestampMetadata` source/frame field splitting and default-reported value behavior.
Improve Basler hardware timestamp robustness by treating support as best-effort, recording the tick-frequency source, falling back to an assumed 1 GHz clock when frequency is unavailable, and ignoring zero-value camera timestamps as missing data. The frame timestamp metadata now includes the frequency source in `extra`, and unused last-frame timestamp state was removed. Video recorder metadata also drops the legacy top-level `timestamps` field in favor of the structured `timestamp_sources` schema.
Updates `test_video_recorder.py` to stop asserting the top-level `timestamps` list in sidecar JSON fixtures. The tests now focus on schema v2 fields that remain authoritative (`frame_timestamps`, `start_time`, `end_time`, `duration_seconds`, and timestamp source metadata), aligning expectations with current sidecar output.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds optional per-frame timestamp metadata (including best-effort Basler hardware timestamps) and threads that metadata through capture, controller signals, recording, and the timestamp sidecar output format.
Changes:
- Introduces
CapturedFrameas the standard return type for camera backendread()calls, carrying software timestamp plus optional timestamp metadata. - Adds
FrameTimestampMetadataand integrates Basler tick-based timestamps (with tick-frequency discovery/fallback) into capture and recording. - Updates
VideoRecordertimestamp sidecar JSON to schema v2 with per-recording timestamp source metadata and per-frame timestamp records.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/utils/test_timestamps.py | Adds unit tests for FrameTimestampMetadata dict splitting and default selection behavior. |
| tests/services/test_video_recorder.py | Updates/extends tests for schema v2 timestamp sidecar output and hardware metadata inclusion. |
| tests/services/test_multicam_controller.py | Adjusts signal handler signatures and adds coverage for forwarding timestamp metadata. |
| tests/gui/test_rec_manager.py | Adds coverage ensuring RecordingManager.write_frame() forwards timestamp metadata to recorder. |
| tests/conftest.py | Updates test backend and recorder fakes to use CapturedFrame and pass through timestamp metadata. |
| tests/cameras/backends/test_opencv_backend.py | Updates OpenCV backend tests for CapturedFrame return type. |
| tests/cameras/backends/test_gentl_trigger.py | Updates GenTL trigger test to read .frame from CapturedFrame. |
| tests/cameras/backends/test_gentl_backend.py | Updates GenTL backend tests for CapturedFrame return type. |
| tests/cameras/backends/test_basler_backend.py | Updates Basler backend tests and adds coverage for Basler timestamp metadata. |
| tests/cameras/backends/test_aravis_backend.py | Updates Aravis backend tests for CapturedFrame return type and minor unused var cleanup. |
| tests/cameras/backends/conftest.py | Extends fake Basler SDK objects to provide timestamp and tick-frequency nodes. |
| dlclivegui/utils/timestamps.py | Introduces FrameTimestampMetadata dataclass and serialization helpers. |
| dlclivegui/services/video_recorder.py | Threads timestamp metadata through queue/write loop and writes schema v2 timestamp sidecar JSON. |
| dlclivegui/services/multi_camera_controller.py | Extends capture/recording signals and internal plumbing to propagate timestamp metadata. |
| dlclivegui/gui/recording_manager.py | Forwards timestamp metadata into VideoRecorder.write(). |
| dlclivegui/gui/main_window.py | Passes timestamp metadata from controller signal into recording manager. |
| dlclivegui/cameras/base.py | Adds CapturedFrame, updates backend interface to return it, and extends capabilities. |
| dlclivegui/cameras/backends/opencv_backend.py | Updates OpenCV backend to return CapturedFrame on success/failure paths. |
| dlclivegui/cameras/backends/gentl_backend.py | Updates GenTL backend to return CapturedFrame. |
| dlclivegui/cameras/backends/basler_backend.py | Adds Basler timestamp tick-frequency handling and per-frame timestamp metadata generation. |
| dlclivegui/cameras/backends/aravis_backend.py | Updates Aravis backend to return CapturedFrame. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds frame capture metadata and recording timestamp support, with best-effort Basler hardware timestamp integration.
This PR refactors camera reads to allow returning extra objects alongside the frame, such as the existing host-side software timestamp, as well as optional backend-provided timestamp metadata.
The software timestamp remains the primary source, while hardware timestamps are added only when the backend can report them.
Main changes
What this does not do yet
Testing
Added and updated tests accordingly; tested and confirmed to work as intended with 1x a2A1920-160umPRO.