cosmo-edge/docs/benchmarks/scenario-bench/v1.1/report.html at main · cosmo-wander-ai/cosmo-edge · GitHub

1
<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>CosmoEdge 1.1 Multi-Platform Video Analytics Benchmark</title><style>:root{color-scheme:light}*{box-sizing:border-box}body{margin:0;background:#f7f9fc;color:#172033;font:15px/1.65 Inter,"Segoe UI",Arial,sans-serif}main{max-width:1120px;margin:auto;padding:42px 24px 70px;min-width:0}h1{font-size:32px;line-height:1.25;margin:0 0 8px;letter-spacing:-.02em}h2{margin-top:38px;line-height:1.35}h3{margin-top:28px}.lead{color:#526071}.notice{background:#eef4ff;border-left:4px solid #2563eb;padding:14px 16px;border-radius:0 8px 8px 0}.release{background:#ecfdf5;border-left-color:#059669}.experimental{background:#fff7e8;border-left-color:#d97706}.report-nav{display:flex;gap:10px;align-items:center;flex-wrap:wrap;margin:0 0 22px}.report-nav a,.report-nav span{display:inline-flex;align-items:center;min-height:34px;padding:5px 11px;border:1px solid #cbd5e1;border-radius:999px;background:#fff;color:#1d4ed8;text-decoration:none}.report-nav span{color:#475569;background:#f8fafc}.report-nav a:hover{border-color:#2563eb;background:#eff6ff}.table{max-width:100%;overflow-x:auto;overscroll-behavior-inline:contain;-webkit-overflow-scrolling:touch;border:1px solid #dce3ed;border-radius:8px;margin:12px 0 24px;background:#fff}.table:focus{outline:2px solid #93c5fd;outline-offset:2px}table{border-collapse:collapse;width:100%;background:#fff}th,td{padding:10px 12px;border-bottom:1px solid #dce3ed;text-align:left;vertical-align:top}th{background:#f1f5f9;white-space:nowrap}tr:last-child td{border-bottom:0}td[data-status="PASS"]{color:#047857;font-weight:700}td[data-status="FAIL"],td[data-status="STOP"]{color:#b91c1c;font-weight:700}img{display:block;max-width:100%;height:auto;margin:24px auto;background:#fff;border:1px solid #e2e8f0;border-radius:10px}code{background:#eef2f7;padding:2px 5px;border-radius:4px}a{color:#1d4ed8}@media(max-width:600px){main{padding:26px 15px 52px}h1{font-size:27px}h2{font-size:22px;margin-top:32px}.table{margin-right:0}.table table{width:max-content;min-width:100%;max-width:none}th,td{padding:9px 11px;min-width:76px}img{margin:18px auto}.report-nav{gap:7px}.report-nav a,.report-nav span{font-size:13px;min-height:32px;padding:4px 9px}}</style></head><body><main><nav class="report-nav" aria-label="Report navigation"><span>CosmoEdge 1.1 release-candidate report</span><a href="report.zh-CN.html">中文</a></nav><h1>CosmoEdge 1.1 Multi-Platform Video Analytics Benchmark</h1><p class="lead">BM1688 · CV186X · RK3576</p><p class="notice">Under the stated models, video, and device configurations, the BM1688 short-run dual-detector staircase completed 16 channels at 5 FPS, while CV186X and RK3576 completed 8. These are highest verified short-run points, not recommended profiles or theoretical chip limits.</p><p class="notice release"><strong>Release-candidate performance material</strong><br>Publication scope: this page and its sanitized attachments have completed repository-level preparation as CosmoEdge 1.1 release-candidate performance material. BM1688/CV186X evidence is bound to the Open package recorded in the manifest; Protected and RK3576 package identities remain product-release evidence and are not distributed with this public benchmark.</p><h2>Recommended profiles and short-run observations</h2><div class="table" tabindex="0" role="region" aria-label="Scrollable data table"><table><tr><th>Platform</th><th>Device</th><th>Recommended profile</th><th>Highest verified point</th><th>Hold / step</th><th>Status</th></tr><tr><td>BM1688</td><td>BM1688 reference device</td><td>Pending repetition and soak</td><td>≥ 16 channels × 2 detectors × 5 FPS</td><td>15 s</td><td>Preliminary</td></tr><tr><td>CV186X</td><td>CV186X reference device</td><td>Pending repetition and soak</td><td>≥ 8 channels × 2 detectors × 5 FPS</td><td>30 s</td><td>Preliminary</td></tr><tr><td>RK3576</td><td>RK3576 EVB</td><td>Pending repetition and soak</td><td>≥ 8 channels × 2 detectors × 5 FPS</td><td>15 s</td><td>Preliminary</td></tr></table></div><img src="assets/capacity-overview.svg" alt="Workload envelope by platform"><img src="assets/throughput-curves.svg" alt="Dual-detector throughput and latency curves"><img src="assets/resource-peaks.svg" alt="Resource peaks at the highest verified point"><h2>Single-detector capacity matrix</h2><p>Values are last passing channels. “≥” means the highest configured point passed. “*” means the next step was blocked by task binding and is not a performance limit.</p><div class="table" tabindex="0" role="region" aria-label="Scrollable data table"><table><tr><th>Platform</th><th>Workload</th><th>24 FPS</th><th>10 FPS</th><th>7 FPS</th><th>5 FPS</th><th>Status</th></tr><tr><td>BM1688</td><td>Person detector</td><td>7</td><td>15</td><td>≥16</td><td>≥16</td><td>Preliminary</td></tr><tr><td>BM1688</td><td>Safety-helmet detector</td><td>7</td><td>≥12*</td><td>≥16</td><td>≥16</td><td>Preliminary</td></tr><tr><td>CV186X</td><td>Person detector</td><td>≥3*</td><td>≥8*</td><td>≥11*</td><td>≥15*</td><td>Preliminary</td></tr><tr><td>CV186X</td><td>Safety-helmet detector</td><td>≥3*</td><td>≥8*</td><td>≥11*</td><td>≥15*</td><td>Preliminary</td></tr><tr><td>RK3576</td><td>Person detector</td><td>6</td><td>12</td><td>15</td><td>≥16</td><td>Preliminary</td></tr><tr><td>RK3576</td><td>Safety-helmet detector</td><td>5</td><td>10</td><td>12</td><td>15</td><td>Preliminary</td></tr></table></div><h2>Experimental: VLM runtime observation</h2><p class="notice experimental">The requested rate is 0.1 FPS per channel, but analysis FPS is excluded from PASS/FAIL. Passing means non-FPS gates only and must not be interpreted as validated VLM capacity. RK3576 reports a shared task counter; this page uses the reviewed equivalent per-channel observations documented in methodology.md.</p><div class="table" tabindex="0" role="region" aria-label="Scrollable data table"><table><tr><th>Platform</th><th>Target FPS / ch</th><th>Last non-FPS pass</th><th>Equivalent FPS at that point</th><th>Stop point</th><th>Status</th></tr><tr><td>BM1688</td><td>0.1</td><td>8</td><td>0.04</td><td>highest configured point observed</td><td>Experimental</td></tr><tr><td>CV186X</td><td>0.1</td><td>8</td><td>0.08</td><td>highest configured point observed</td><td>Experimental</td></tr><tr><td>RK3576</td><td>0.1</td><td>7</td><td>0.063</td><td>Channel 8 non-FPS stop</td><td>Experimental</td></tr></table></div><h2>Linked attachments</h2><p>The primary report keeps the public conclusions concise; these sanitized attachments provide per-channel evidence and machine-readable results.</p><div class="table" tabindex="0" role="region" aria-label="Scrollable data table"><table><tr><th>Platform</th><th>Single detector</th><th>Dual detector</th><th>VLM</th><th>Machine-readable data</th></tr><tr><td><a href="results/bm1688/report.html">BM1688</a></td><td><a href="results/bm1688/single-detector/report.html">Single-detector staircase</a></td><td><a href="results/bm1688/dual-detector/report.html">Dual-detector staircase</a></td><td><a href="results/bm1688/vlm-observation/report.html">VLM observation</a></td><td><a href="results/bm1688/summary.json">summary.json</a> · <a href="results/bm1688/metrics.json">metrics.json</a></td></tr><tr><td><a href="results/cv186x/report.html">CV186X</a></td><td><a href="results/cv186x/single-detector/report.html">Single-detector staircase</a></td><td><a href="results/cv186x/dual-detector/report.html">Dual-detector staircase</a></td><td><a href="results/cv186x/vlm-observation/report.html">VLM observation</a></td><td><a href="results/cv186x/summary.json">summary.json</a> · <a href="results/cv186x/metrics.json">metrics.json</a></td></tr><tr><td><a href="results/rk3576/report.html">RK3576</a></td><td><a href="results/rk3576/single-detector/report.html">Single-detector staircase</a></td><td><a href="results/rk3576/dual-detector/report.html">Dual-detector staircase</a></td><td><a href="results/rk3576/vlm-observation/report.html">VLM observation</a></td><td><a href="results/rk3576/summary.json">summary.json</a> · <a href="results/rk3576/metrics.json">metrics.json</a></td></tr></table></div><h2>Comparability</h2><p>BM1688 and CV186X use the same byte-identical Open Sophon detector/classifier artifacts. RK3576 uses platform-specific RKNN artifacts that are not proven to be equivalent conversions of the same frozen source checkpoint and precision. This report describes typical workloads and does not rank chips.</p><h2>Test environment</h2><div class="table" tabindex="0" role="region" aria-label="Scrollable data table"><table><tr><th>Platform</th><th>Board</th><th>OS</th><th>Runtime / media</th><th>CosmoEdge</th></tr><tr><td>BM1688</td><td>Sophon BM1688 reference design; commercial enclosure name withheld</td><td>Ubuntu 22.04.5 LTS</td><td>inference: libsophon/BMRT 0.4.12; media: Sophon FFmpeg 2.0.0 / Sophon GStreamer 2.0.0; decoder: bmvpu-decoder; encoder: bmvpu-h264</td><td>V1.1.0.0</td></tr><tr><td>CV186X</td><td>Sophon CV186X reference design; commercial enclosure name withheld</td><td>Ubuntu 22.04.5 LTS</td><td>inference: libsophon/BMRT 0.4.12; media: Sophon FFmpeg 2.0.0 / Sophon GStreamer 2.0.0; decoder: bmvpu-decoder; encoder: bmvpu-h264</td><td>V1.1.0.0</td></tr><tr><td>RK3576</td><td>Rockchip RK3576 EVB1 V10 Board</td><td>Linux distribution not exposed by the read-only device API</td><td>inference: RKNN runtime version not exposed by the read-only device API; driver: RKNN driver version not exposed by the read-only device API; rga: version not exposed; runtime telemetry confirms RGA path; mpp: version not exposed; runtime telemetry confirms Rockchip MPP VPU; decoder: rockchip-mpp-vpu / copy-out I420; encoder: rockchip-mpp / copy-first stride-aligned I420</td><td>V1.1.0.0</td></tr></table></div><h2>Method and reproduction</h2><ol><li>Fixed input: H.264, 1920×1080, 24 FPS; SHA-256 is in release-manifest.json.</li><li>Add one channel per step; sample CV about every 3 seconds and VLM about every 10 seconds; use the second half as steady state.</li><li>See models/ for model identities, scenarios/ for public descriptors, and results/ for per-step data.</li><li>Resolve the descriptor locally and follow methodology.md; the public pack contains no device addresses, credentials, or internal IDs.</li></ol><h2>Limitations</h2><ul><li>A short staircase is not a soak test and does not create a recommended profile.</li><li>This is the release-candidate public performance report; it does not replace full installation, upgrade, rollback, accuracy, or soak qualification.</li><li>VLM has no FPS gate and remains an Experimental runtime observation.</li><li>Model accuracy qualification is outside this capacity report.</li><li>The open detector/classifier files are present in the BM1688 and CV186X platform resource sets; the sample video and other model binaries are not redistributed by this benchmark.</li></ul><p>Full contract: <code>release-manifest.json</code> · <code>methodology.md</code> · <code>SHA256SUMS</code></p></main></body></html>