Turn video AI models into deployable edge applications — a C++ edge AI engine for Sophon, Rockchip, and x86.
Build and operate video analytics, VLM, and event workflows through a consistent orchestration experience. Each platform uses its own runtime, build, and model artifacts.
Quick Start · Platforms · Validation · Documentation · 简体中文
hero1_compressed.mp4
CosmoEdge goes beyond model serving with a complete application layer for model import, visual orchestration, alarms, and event delivery. The core engine and console in this repository are released under Apache-2.0; certified hardware, commercial preset models, and Model Guard distribution protection have separate boundaries.
This branch is the CosmoEdge 1.1 source line. Platform capacity varies by model and workload; use the linked evidence rather than treating a short-run maximum as a deployment recommendation.
- Multi-platform release: BM1688, CV186X, and RK3576 use the same video-ingest, orchestration, event, and observability workflow with target-specific model artifacts.
- Public benchmark pack: the CosmoEdge 1.1 multi-platform report covers single-detector, dual-detector, and Experimental VLM workloads with sanitized reproducibility attachments.
- Rockchip RK3576: cross-build, board operation, RKNN inference, and MPP/RGA media paths are integrated. A 4-channel single-detector profile has 12-hour evidence; higher short-run points remain measured boundaries rather than recommended profiles.
- Sophon model handling: chip-aware validation supports target-specific
.nnartifacts for BM1688 and CV186X. The benchmark records an exact Open-package and running-engine binding for both reference devices. - RKNN data path: targeted DMA-BUF-to-RGA input, persistent bound-input, native quantized output, and direct YOLOv8 tensor decoding paths with explicit fallbacks.
- Agent-assisted development: a repository-guided path for handing model porting, integration, and UI tasks to the coding agent you already use and receiving verifiable deliverables.
- Model Guard 2.3: protects commercial preset-model distribution in Sophon Protected packages. Open and Protected expose the same application features, with no SKU-gated software functionality; they differ in model encryption and device-provisioning tooling.
- macOS Docker Preview: an isolated
linux/amd64candidate for one local-video workflow on Apple Silicon. Two consecutive end-to-end acceptance runs are still required before publication; it does not enable Model Guard or represent native/NPU performance.
CosmoEdge provides one engine architecture and orchestration experience, but each build selects one inference backend and uses models generated for that target platform.
| Platform | Status | Runtime / model artifact | Current scope and evidence |
|---|---|---|---|
| Sophon BM1688 | v1.1 supported / primary | BMRT / .nn |
Production deployment path with v1.1 workload evidence and published v1.0 baselines |
| Rockchip RK3576 | v1.1 supported | RKNN / .rknn |
Cross-build and board paths validated; see the integration guide and v1.1 workload evidence |
| Sophon CV186X | v1.1 supported | BMRT / target-specific .nn |
Model import and device workload evidence included in the v1.1 benchmark |
| x86 Linux / Windows; Apple Silicon macOS | Linux / Windows supported; macOS Preview | ONNX Runtime / .onnx |
Mac uses amd64 emulation for one local-video developer workflow, not native performance evidence |
| Sophon BM1684X | Planned | — | Not part of the current release scope |
No edge hardware is required. The x86 mode uses the same UI and workflow with lower throughput than an NPU deployment.
# 1. Clone
git clone https://github.com/cosmo-wander-ai/cosmo-edge.git
cd cosmo-edge
# 2. Start on Linux
sudo docker compose -f docker-compose.x86.yml up -d --build
# Windows: docker compose -f docker-compose.x86.windows.yml up -d --build
# 3. Open http://localhost:8080Apple Silicon macOS uses a separate amd64-emulation Preview path:
./scripts/macos-docker-preview.sh doctor
./scripts/macos-docker-preview.sh up
./scripts/macos-docker-preview.sh status
# Open http://127.0.0.1:8080The Mac Preview is for local, single-video evaluation. It is not a native macOS binary, an NPU deployment, or production performance evidence. Read the full macOS Docker Preview boundaries first.
After startup, use the Scenario Configuration tutorial to create your first AI detection task. Docker Compose V1 users can replace docker compose with docker-compose.
git clone https://github.com/cosmo-wander-ai/cosmo-edge.git
cd cosmo-edge
# BM1688 (default when the chip model is omitted)
./scripts/docker-compose.sh -f docker-compose.sophon.yml run --rm cosmo-sophon-package --chip bm1688
# CV186X
./scripts/docker-compose.sh -f docker-compose.sophon.yml run --rm cosmo-sophon-package --chip cv186xThe wrapper selects the available Compose implementation and requests elevated
Docker access only when required. The build selects the matching model resources
without a model-path argument and exports each target independently under
build_output/public-runtime/<chip>/, together with TARGET_CHIP and
SHA256SUMS.
The default Open package contains plaintext models and requires no device authorization. Verify the chip marker and checksum before deployment. The Build Guide is the authoritative build reference; follow the Deployment Guide for SSH installation, web upgrade, recovery, and post-reboot acceptance.
./scripts/docker-compose.sh -f docker-compose.rockchip.yml pull cosmo-rockchip-package
# RK3576 (uses the tracked RK3576 model resources)
COSMO_TARGET_CHIP=rk3576 ./scripts/docker-compose.sh \
-f docker-compose.rockchip.yml run --rm cosmo-rockchip-package
ls -lh build_output/rk3576/
# RV1126B (requires a prepared target model overlay)
COSMO_TARGET_CHIP=rv1126b ./scripts/docker-compose.sh \
-f docker-compose.rockchip.yml run --rm cosmo-rockchip-package
ls -lh build_output/rv1126b/One digest-pinned Rockchip builder shares the compiler and RKNN SDK while
selecting an isolated MPP/RGA profile for each chip. RKLLM v1.3.0 is required
and packaged only for RK3576. Target markers, media-profile identities, and
checksums are exported under build_output/<chip>/. See the
Build Guide and
RK3576 integration guide for model
and device-validation boundaries.
For CV186X, follow the CV186X Quick Start for package installation, model import, first-event verification, upgrade, and recovery boundaries.
- Real-time video analytics: detection, classification, tracking, zones, counters, OSD, and alarm snapshots.
- Prompt-driven vision: VLM state judgment and GroundingDINO open-vocabulary detection alongside conventional CV pipelines.
- Visual application workflows: connect models, rules, events, and output actions in the browser.
- Edge integrations: operate managed tasks and deliver structured events through REST, WebSocket, MQTT, or HTTP webhooks.
| Capability | What it covers | Go deeper |
|---|---|---|
| Native runtime | C++17 engine for multi-channel media, inference scheduling, OSD, tasks, and events | Architecture |
| Visual orchestration | Browser-based pipeline composition, task binding, parameter validation, and live feedback | Pipeline tutorial |
| Inference and media | Platform backends for Sophon, RKNN, and x86; platform-specific builds and model artifacts | Build Guide |
| VLM and DINO | Prompt-based judgment, open-vocabulary detection, and optional VLM review before a detection alarm is reported | VLM Guide |
| Operations and integration | Model management, alarms, event history, REST, WebSocket, MQTT, and webhooks | API Overview |
| Model onboarding and protection | Model conversion, import, validation, and the Open/Protected distribution boundary | Model Porting Guide |
▶ Watch: compose a complete visual pipeline
pipline2_compressed.mp4
▶ Watch: GroundingDINO and VLM visual workflows
promot1_compressed.mp4
Already have a model-porting, integration, or UI task? Give your usual coding agent the business goal, available materials, target device or test environment, and acceptance criteria. The repository provides task entry points, examples, checks, and evidence boundaries so the result can include importable artifacts, scoped code changes, and a verifiable conclusion.
Start with Agent-Assisted Development, then use the Model Porting Guide or Contributor Guide for the task at hand.
The publication-ready v1.1 material covers BM1688, CV186X, and RK3576 with a concise public report plus linked single-detector, dual-detector, and Experimental VLM attachments. Results are short-run workload evidence unless a longer duration is explicitly stated; they are not theoretical chip limits or automatic deployment recommendations.
- English benchmark index
- 中文基准报告索引
- English primary report (rendered documentation site)
- Methodology and reproduction
The release-branch preparation baseline is feat/model-guard-v2.3 commit 209bc2b52849864a15bdad91beb61f5bc982c17f, tree f64a98bce05b9ee8dc64dda8e56ad50f9d15687f. BM1688 and CV186X are byte-bound to the recorded Open package and running engine; that package does not embed a source commit and therefore is not claimed as a reproducible build of the preparation baseline. The RK3576 package digest and Protected-package provenance are not distributed in this public benchmark, so the report does not make package-qualified claims for them. See the release manifest for the exact evidence boundary.
The following table preserves the previously published v1.0 baseline for historical comparison:
| ScenarioBench workload | Hardware | Max verified channels | Target FPS | Result | Evidence |
|---|---|---|---|---|---|
| No Safety Helmet | YY-16T01-Preview / NPU | 16 | 3/channel | PASS | report |
| Pedestrian Detection | YY-16T01-Preview / NPU | 16 | 5/channel | PASS | report |
| Pedestrian + No Safety Helmet | YY-16T01-Preview / NPU | 16 | 3/channel/task | PASS | report |
| VLM Review | YY-16T01-Preview / NPU | 8 | 0.1/channel | PASS | report |
| No Safety Helmet x86 baseline | x86 CPU | 7 | 3/channel | LIMITED; 8 channels exceeded latency limits | report |
See the v1.0 benchmark manifest, environment notes, and current refresh notes for the legacy methodology and publication boundaries.
This earlier long-duration evidence remains bound to source 8f8b4b8e (tree fd1b646f), engine SHA-256 bc829d9513334c4520fad1b58439bb3e6e31338c664e93eb15babdaaa564d886, RKNN Runtime 2.3.2-429f97ae6b, driver 0.9.8, RGA 1.10.1_[4], and MPP 1.5.0-1.
| ScenarioBench workload | Hardware | Max verified channels | Target FPS | Result |
|---|---|---|---|---|
| No Safety Helmet | Rockchip RK3576 EVB1 V10 / RKNN | 8 | 5/channel | PASS |
The 1 / 2 / 4 / 8-channel steps all passed with zero discarded frames or inference, RGA, or MPP failures. Single-channel Detect average decreased from 142.3 ms to 58.2 ms (-59.1%). A separate 12-hour run at 4 channels and 5 FPS, with four algorithm previews, recorded 720 continuous hold samples, CPU avg/P95/max of 33.92% / 39% / 43%, and zero discarded frames or runtime/preview failures.
The four-channel, 5 FPS profile completed the 12-hour run. The 8-channel step records engineering headroom and is not a release capacity guarantee. This historical evidence is retained as long-duration context and is not silently rebound to the newer v1.1 source baseline.
+------------------------------------------------------------------+
| Web Console | Visual Orchestration | REST / WebSocket / MQTT |
+--------------------------------+---------------------------------+
|
+--------------------------------v---------------------------------+
| C++ Engine Core |
| Media | Inference | Tasks | Rules | Alarms | Events | Models |
+--------------------------------+---------------------------------+
|
+--------------------------------v---------------------------------+
| Inference and Media Backend Interfaces |
+--------------------+----------------------+----------------------+
| Sophon BMRT/VPU | RKNN + MPP/RGA | ONNX Runtime/FFmpeg |
| BM1688; CV186X | RK3576 | x86 Linux / Windows |
+--------------------+----------------------+----------------------+
One build selects one inference backend. Model artifacts are generated for the target platform, and feature/model coverage and capacity remain platform-specific. Model Guard Protected distribution currently belongs to the Sophon packaging path.
| Start here | Best for |
|---|---|
| Documentation Home | Full documentation index and learning path |
| Quick Start Guide | First setup and scenario run |
| Scenario Configuration | Building scene-level workflows |
| VLM Guide | Prompt-based visual judgment and events |
| Model Porting Guide | Importing your own model |
| Agent-Assisted Development | Delegating an extension task with verifiable results |
| Build Guide | x86, Sophon, and RK3576 build/package paths |
| API Overview | REST, WebSocket, MQTT, and webhook integration |
Certified devices add preconfigured acceleration, validated commercial model packages, and dedicated deployment support; they do not unlock separate software features. Devices are available in mainland China from the Taobao store; contact hello@cosmowander.ai for other regions or project support.
Contributions are welcome through scoped bug reports, documentation improvements, scenarios, and integration notes. Read CONTRIBUTING.md before opening a pull request. Community support is available through GitHub Discussions and Gitee Issues; report vulnerabilities through SECURITY.md.
Can I try CosmoEdge without Sophon or Rockchip hardware?
Yes. Use x86 developer mode on Linux or Windows. Apple Silicon Macs can use the Docker Preview for the console, pipeline, model-management, and integration workflow with one local video. The Mac path uses amd64 emulation and does not enable Model Guard. Edge NPU hardware is still required for target-platform acceleration and capacity validation.
What is the boundary between the Open and Protected packages?
They expose the same application features and use the same MD5 upgrade lifecycle. Open uses plaintext models without device authorization; Sophon Protected packages can carry encrypted commercial preset models and provisioning tooling that require a device-bound certificate. Application archives themselves are not signed.
Can I use my own trained models?
Yes. Use the model-porting path to validate the tensor, preprocessing, post-processing, target runtime, and business accuracy contract. A model artifact must be generated for the platform where it will run.
How production-ready is CosmoEdge?
v1.1.0 is the multi-platform release line for BM1688, CV186X, RK3576, and x86. The linked report records measured workload boundaries and explicit evidence gaps; production sizing still requires validation with your own models, streams, accuracy requirements, and deployment conditions.
CosmoEdge is licensed under the Apache License 2.0. Copyright 2026 CosmoEdge Contributors.
An open-source project by Cosmo Wander AI and the CosmoEdge contributors.
Turn video AI models into deployable edge applications.
📦 This repository is mirrored read-only to Gitee for mainland China access. See MIRRORING.md.