Skip to content

Commit 6d15e1a

Browse files
committed
test(rknn): add full-load qualification gates
Signed-off-by: samuel--hu <genius1@qq.com>
1 parent 2f6216e commit 6d15e1a

11 files changed

Lines changed: 535 additions & 37 deletions

File tree

docs/en/guide/rk3576-rknn-development.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,9 @@ python tools/rknn/prepare_validation_data.py \
123123
--output-dir yolov8-calibration --samples 32
124124

125125
python tools/rknn/convert_model.py \
126-
--spec config/rknn/models/yolov8.json --model yolov8-heads.onnx \
126+
--spec config/rknn/models/yolov8.json \
127+
--platform-profile config/rknn/platforms/rk3576.json \
128+
--model yolov8-heads.onnx \
127129
--output yolov8-heads-int8.rknn --quantize \
128130
--dataset yolov8-calibration/dataset.txt
129131
```

docs/en/tutorials/05-model-porting/model-porting.md

Lines changed: 33 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,14 +14,14 @@ next: false
1414
| Who this is for | ML engineers and integration developers bringing a custom detector or classifier to CosmoEdge |
1515
| What you will accomplish | Evaluate runtime compatibility, convert and upload a model, configure parsing, and complete image, video, and sustained-run validation |
1616
| Prerequisites | Understand Pipelines and know the model input, output, preprocessing, postprocessing, and label order |
17-
| Estimated time | About 40–60 minutes for x86 ONNX; Sophon conversion commonly adds 30–60 minutes |
18-
| Device required | x86 requires an ONNX Runtime CosmoEdge build; Sophon requires a BM1688/CV186X device and matching conversion toolchain |
17+
| Estimated time | About 40–60 minutes for x86 ONNX; Sophon or Rockchip conversion commonly adds 30–60 minutes |
18+
| Device required | x86 requires an ONNX Runtime CosmoEdge build; Sophon and Rockchip require the actual target device and matching conversion toolchain |
1919
| Final acceptance result | The model loads, its output is parsed correctly, image and video results pass, and it runs without resource failure on the target device |
2020

2121
Complete third-party integration in this order:
2222

2323
1. Confirm support conditions and the model contract.
24-
2. Export ONNX; for Sophon, convert it again into a chip-specific `bmodel`.
24+
2. Export ONNX; convert it into a chip-specific `bmodel` for Sophon or `rknn` for Rockchip.
2525
3. Validate the artifact on the conversion host.
2626
4. Upload and configure the model.
2727
5. Run positive and negative image tests first.
@@ -39,12 +39,14 @@ output layout, and postprocessing are compatible with CosmoEdge.
3939
| --- | --- | --- | --- | --- |
4040
| x86 CPU | `.onnx` | `model.onnx` | ONNX Runtime CPU | x86_64 host and matching CosmoEdge build |
4141
| Sophon | `.bmodel` | `model.nn` | Sophon BMRT | BM1688 or CV186X; the artifact must target the actual chip |
42+
| Rockchip RKNN | `.rknn` | `model.rknn` | RKNN Runtime | RK3576 or RV1126B; the artifact must target the actual chip |
4243

4344
`model.nn` is the internal file name in a CosmoEdge model package. It wraps the device model. When adding
4445
an individual Sophon model in the UI, select its `.bmodel`; do not rename an extension to `.nn`.
4546

4647
PyTorch `.pt`, TensorFlow SavedModel, and other training-framework artifacts cannot be uploaded directly.
47-
Export them to ONNX first. Sophon deployments then convert ONNX into a chip-specific `.bmodel`.
48+
Export them to ONNX first. Sophon deployments then convert ONNX into a chip-specific `.bmodel`, while
49+
Rockchip deployments produce a chip-specific `.rknn`. RK3576 and RV1126B `.rknn` artifacts are not interchangeable.
4850

4951
### 1.2 Contracts Beyond the File Format
5052

@@ -66,7 +68,7 @@ or runtime code.
6668
### 1.3 Verified Capability vs Conditional Compatibility
6769

6870
- **Directly supported by current code**: Add `.onnx` on x86, add `.bmodel` on Sophon, and import packages
69-
containing `model.onnx` or `model.nn`.
71+
containing `model.onnx` or `model.nn`; RKNN builds add `.rknn` and package it as `model.rknn`.
7072
- **Reference evidence in this repository**: a YOLOv8 detector has completed x86 ONNX import, live overlay,
7173
and event output.
7274
- **Still required on the target candidate**: validate your exact model, Sophon artifact, performance,
@@ -387,6 +389,32 @@ task acceptance, but it must not borrow another example's fixed shapes or hashes
387389

388390
The Sophon Add Model page requires a `.bmodel` file.
389391

392+
## Rockchip RKNN Path: Shared Backend, Target-Specific Artifacts
393+
394+
RK3576 and RV1126B share one CosmoEdge RKNN inference implementation, Rockchip media interface, and model
395+
contract. Chip differences come from `config/rknn/platforms/<chip>.json`. A model spec does not hard-code a
396+
chip, but every conversion binds one platform profile, so the resulting `.rknn` remains target-specific and
397+
must not be copied between chips.
398+
399+
The agent-assisted flow always enters through `scripts/agent/convert_model.sh` and `verify.sh`. It selects
400+
RKNN Toolkit2 from the task contract and freezes Python, wheel, platform profile, model spec, calibration
401+
set, and artifact hashes. RK3576 and RV1126B do not maintain separate conversion scripts. For manual
402+
diagnosis, follow the [RK3576 RKNN development guide](/en/guide/rk3576-rknn-development) and select the
403+
actual target profile:
404+
405+
```bash
406+
python tools/rknn/convert_model.py \
407+
--spec config/rknn/models/yolov8.json \
408+
--platform-profile config/rknn/platforms/rv1126b.json \
409+
--model yolov8-heads.onnx \
410+
--output yolov8-rv1126b-int8.rknn \
411+
--quantize --dataset calibration/dataset.txt
412+
```
413+
414+
Conversion-host success is not device acceptance. Validate the matching Runtime/driver, numerical output,
415+
image and video postprocessing, OSD, rules, alerts, the 5 FPS target, and stability on the actual device,
416+
with results bound to the target chip and artifact SHA-256.
417+
390418
## 4. Upload and Configure the Model
391419

392420
### 4.1 Open Model Repository

docs/guide/rk3576-rknn-development.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -107,7 +107,9 @@ python tools/rknn/prepare_validation_data.py \
107107
--output-dir yolov8-calibration --samples 32
108108

109109
python tools/rknn/convert_model.py \
110-
--spec config/rknn/models/yolov8.json --model yolov8-heads.onnx \
110+
--spec config/rknn/models/yolov8.json \
111+
--platform-profile config/rknn/platforms/rk3576.json \
112+
--model yolov8-heads.onnx \
111113
--output yolov8-heads-int8.rknn --quantize \
112114
--dataset yolov8-calibration/dataset.txt
113115
```

docs/tutorials/05-model-porting/model-porting.md

Lines changed: 30 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,14 +14,14 @@ next: false
1414
| 适合谁 | 需要把自有检测或分类模型接入 CosmoEdge 的算法工程师和集成开发者 |
1515
| 完成后能做什么 | 判断模型是否满足运行条件,转换并上传模型,配置解析参数,完成图片、视频和持续运行验证 |
1616
| 使用前提 | 已理解 Pipeline;掌握模型输入、输出、预处理、后处理和标签顺序 |
17-
| 预计时间 | x86 ONNX 路径约 40–60 分钟;Sophon 转换路径通常需要额外 30–60 分钟 |
18-
| 是否需要设备 | x86 路径需要 ONNX Runtime 版 CosmoEdge;Sophon 路径需要 BM1688/CV186X 设备及匹配转换工具链 |
17+
| 预计时间 | x86 ONNX 路径约 40–60 分钟;Sophon 或 Rockchip 转换路径通常需要额外 30–60 分钟 |
18+
| 是否需要设备 | x86 路径需要 ONNX Runtime 版 CosmoEdge;Sophon 与 Rockchip 路径需要目标芯片设备及匹配转换工具链 |
1919
| 最终验收结果 | 模型可加载、推理输出可解析、图片与视频结果正确,并在目标设备上持续运行无资源错误 |
2020

2121
第三方模型接入按以下顺序完成:
2222

2323
1. 确认支持条件和模型契约。
24-
2. 导出 ONNX;Sophon 设备再转换为目标芯片的 `bmodel`
24+
2. 导出 ONNX;Sophon 设备转换为目标芯片的 `bmodel`,Rockchip 设备转换为目标芯片的 `rknn`
2525
3. 在转换主机上检查产物。
2626
4. 上传并配置模型。
2727
5. 先做正负图片验证。
@@ -39,12 +39,14 @@ next: false
3939
| --- | --- | --- | --- | --- |
4040
| x86 CPU | `.onnx` | `model.onnx` | ONNX Runtime CPU | x86_64 主机和对应 CosmoEdge 构建 |
4141
| Sophon | `.bmodel` | `model.nn` | Sophon BMRT | BM1688 或 CV186X,转换产物必须匹配芯片 |
42+
| Rockchip RKNN | `.rknn` | `model.rknn` | RKNN Runtime | RK3576 或 RV1126B,转换产物必须匹配实际芯片 |
4243

4344
`model.nn` 是 CosmoEdge 模型包中的内部文件名,封装的是设备侧模型;通过页面单独添加
4445
Sophon 模型时应选择 `.bmodel`,不要把文件扩展名手工改成 `.nn`
4546

4647
PyTorch `.pt`、TensorFlow SavedModel 或其他训练框架产物不能直接上传。它们必须先导出
47-
为 ONNX;Sophon 还要使用匹配工具链把 ONNX 转为目标芯片的 `.bmodel`
48+
为 ONNX;Sophon 还要使用匹配工具链把 ONNX 转为目标芯片的 `.bmodel`,Rockchip 则转换
49+
为目标芯片的 `.rknn`。RK3576 与 RV1126B 的 `.rknn` 不是可互换模型。
4850

4951
### 1.2 格式之外还要匹配的契约
5052

@@ -65,7 +67,7 @@ CosmoEdge 当前已有 `YOLOV8_DET` 等解析路径,但“任意 ONNX”并不
6567
### 1.3 已验证能力与条件性兼容
6668

6769
- **由当前代码直接支持**:x86 添加 `.onnx`、Sophon 添加 `.bmodel`,以及模型包中的
68-
`model.onnx` / `model.nn`
70+
`model.onnx` / `model.nn`;RKNN 构建添加 `.rknn`,模型包中使用 `model.rknn`
6971
- **仓库中已有参考证据**:YOLOv8 检测模型在 x86 ONNX 路径完成过模型导入、实时叠加
7072
和事件输出。
7173
- **仍需在目标候选版本上验证**:你的具体模型、Sophon 转换产物、性能、资源占用、
@@ -366,6 +368,29 @@ model_tool --info yolov8n_bm1688_f16.bmodel
366368

367369
Sophon 添加模型页面会要求 `.bmodel` 文件,实际页面见下一节的添加模型表单。
368370

371+
## Rockchip RKNN 路径:共享后端、目标专用产物
372+
373+
RK3576 与 RV1126B 共用同一套 CosmoEdge RKNN 推理实现、Rockchip 媒体接口和模型契约;
374+
芯片差异由 `config/rknn/platforms/<chip>.json` 平台 profile 提供。模型 spec 不写死芯片,
375+
但每次转换必须绑定一个 profile,因此输出的 `.rknn` 仍是目标专用产物,不能跨芯片复制使用。
376+
377+
智能体辅助流程统一使用 `scripts/agent/convert_model.sh``verify.sh`。执行器按任务合同选择
378+
RKNN Toolkit2,冻结 Python、wheel、平台 profile、模型 spec、校准集和产物哈希。RK3576 与
379+
RV1126B 不各自维护一套转换脚本。手工排障时可参考
380+
[RK3576 RKNN 开发说明](/guide/rk3576-rknn-development),并把平台参数换成实际目标:
381+
382+
```bash
383+
python tools/rknn/convert_model.py \
384+
--spec config/rknn/models/yolov8.json \
385+
--platform-profile config/rknn/platforms/rv1126b.json \
386+
--model yolov8-heads.onnx \
387+
--output yolov8-rv1126b-int8.rknn \
388+
--quantize --dataset calibration/dataset.txt
389+
```
390+
391+
转换主机通过不等于设备验收。必须在对应设备上继续验证 Runtime/驱动、数值输出、图片与视频
392+
后处理、OSD、规则、告警、5 FPS 目标以及稳定性;这些结果应绑定目标芯片和产物 SHA-256。
393+
369394
## 4. 上传并配置模型
370395

371396
### 4.1 进入模型仓库

tools/scenario-bench/README.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -76,11 +76,15 @@ node src/cli.js checkpoint \
7676
--max-gap-sec 120 \
7777
--min-fps-ratio 0.9 \
7878
--expected-preview-streams 4 \
79-
--expected-decoder-backend rockchip-copy-out \
80-
--expected-encoder-backend rockchip-copy-first
79+
--expected-decoder-backend rockchip-mpp-rga \
80+
--expected-encoder-backend rockchip-mpp-rga \
81+
--min-rga-bound-native-int8-frames 1 \
82+
--min-rknn-mpp-dmabuf-frames 1 \
83+
--min-rknn-rga-crop-dmabuf-frames 1 \
84+
--max-rga-bound-requantize-avg-ms 2
8185
```
8286

83-
输出结论只有三种:`IN_PROGRESS` 表示时长尚未达到且其余门禁正常(命令退出码 3),`FAIL` 表示连续性、负载、资源或原生媒体链路存在失败(退出码 1),`PASS` 仅在时长达到且全部必需检查通过时产生(退出码 0)。审计覆盖采样间断/新鲜度、固定路数、逐通道 FPS 与丢帧、CPU/内存/磁盘、内存池增长、MPP/RGA/RKNN 失败及计数器回退、延迟 Copy-out 记账、OSD/编码/发布帧流和 SRS 发布流;输入和候选身份文件的 SHA-256 会写入结果。
87+
输出结论只有三种:`IN_PROGRESS` 表示时长尚未达到且其余门禁正常(命令退出码 3),`FAIL` 表示连续性、负载、资源或原生媒体链路存在失败(退出码 1),`PASS` 仅在时长达到且全部必需检查通过时产生(命令退出码 0)。`gate-hours` 从首次观测到目标路数、活动路数和唯一通道数均达到配置负载时开始累计;登录、建连和部分路数爬坡只计入 `runAgeSec`,不计入验收使用的 `fullLoadCoverageSec`审计覆盖采样间断/新鲜度、固定路数、逐通道 FPS 与丢帧、CPU/内存/磁盘、内存池增长、MPP/RGA/RKNN 失败及计数器回退、延迟 Copy-out 记账、OSD/编码/发布帧流和 SRS 发布流;针对原生 RKNN 路径还可要求 MPP DMA-BUF 输入、分类裁剪 DMA-BUF、全部 forward 的 INT8 直绑覆盖和必要符号位变换的平均耗时。输入和候选身份文件的 SHA-256 会写入结果。
8488

8589
## 创建场景包
8690

tools/scenario-bench/src/cli.js

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ init-scenario options:
109109
checkpoint options:
110110
--input <file> Running metrics.partial.json or completed metrics.json
111111
--output <file> Atomic JSON checkpoint output path
112-
--gate-hours <n> Required continuous runtime, e.g. 12
112+
--gate-hours <n> Required observed full-load runtime; excludes login/ramp, e.g. 12
113113
--identity <file> Optional immutable candidate identity key=value file
114114
--source-label <s> Canonical remote/source path recorded in evidence
115115
--identity-label <s> Canonical identity path recorded in evidence
@@ -121,6 +121,10 @@ checkpoint options:
121121
--expected-decoder-backend <s> Required decoder backend identifier
122122
--expected-encoder-backend <s> Required encoder backend identifier
123123
--min-rga-bound-uint8-frames <n> Required fused RGA-to-RKNN UINT8 frames
124+
--min-rga-bound-native-int8-frames <n> Required native INT8 bound frames with full RKNN accounting
125+
--min-rknn-mpp-dmabuf-frames <n> Required RKNN frames sourced from MPP DMA-BUF
126+
--min-rknn-rga-crop-dmabuf-frames <n> Required classifier crops sourced from DMA-BUF
127+
--max-rga-bound-requantize-avg-ms <n> Maximum average native U8-to-INT8 transform latency
124128
`);
125129
}
126130

@@ -751,6 +755,10 @@ async function runCheckpoint(args) {
751755
maxPoolGrowthBytes: optionalNumber(args, 'max-pool-growth-bytes'),
752756
poolGrowthWarmupSec: optionalNumber(args, 'pool-growth-warmup-sec'),
753757
minRgaBoundUint8Frames: optionalNumber(args, 'min-rga-bound-uint8-frames'),
758+
minRgaBoundNativeInt8Frames: optionalNumber(args, 'min-rga-bound-native-int8-frames'),
759+
minRknnMppDmaBufFrames: optionalNumber(args, 'min-rknn-mpp-dmabuf-frames'),
760+
minRknnRgaCropDmaBufFrames: optionalNumber(args, 'min-rknn-rga-crop-dmabuf-frames'),
761+
maxRgaBoundRequantizeAvgMs: optionalNumber(args, 'max-rga-bound-requantize-avg-ms'),
754762
expectedPreviewStreams: optionalNumber(args, 'expected-preview-streams'),
755763
expectedDecoderBackend: args['expected-decoder-backend'],
756764
expectedEncoderBackend: args['expected-encoder-backend'],

0 commit comments

Comments
 (0)