Hello.
My original ONNX model was NCHW.
My goal is to get rid of client side image-transposing by making the transpose operation a part of the bmodel itself.
We have a yolov8m based model, and the following snippet has been modified from the official sophon-demo yolov8_plus_det app. It works perfectly, with no issues.
model_transform.py \
--model_name $model_name \
--model_def $onnx_path \
--input_shapes [[$1,3,352,640]] \
--mean 0.0,0.0,0.0 \
--scale 0.0039216,0.0039216,0.0039216 \
--keep_aspect_ratio \
--pixel_format rgb \
--mlir ${model_name}_$1b.mlir
The problem arises when I try to transpose (notice --input_shapes and --channel_format:
model_transform.py \
--model_name $model_name \
--model_def $onnx_path \
--input_shapes [[$1,352,640,3]] \
--channel_format nhwc \
--mean 0.0,0.0,0.0 \
--scale 0.0039216,0.0039216,0.0039216 \
--keep_aspect_ratio \
--pixel_format rgb \
--mlir ${model_name}_$1b.mlir
In this case, the MLIR output gets generated just fine, but the model_deploy script fails.
model_deploy.py \
--mlir ${model_name}_$1b.mlir \
--quantize F32 \
--chip $target \
--model ${model_name}_fp32_$1b.bmodel
[Running]: tpuc-opt yolov8m_bm1684x_f32_final.mlir --codegen="model_file=yolov8m_fp32_1b.bmodel embed_debug_info=False model_version=latest bmodel_only=False gdma_check=True" -o /dev/null
The dir path of compiler_profile is "./"
bmcpu init: skip cpu_user_defined
Cannot open libusercpu.so, disable user cpu layer.
size does not match for tensor model.0.conv.weight
readTensor failed
UNREACHABLE executed at /__w/tpu-mlir/tpu-mlir/lib/Support/TensorFile.cpp:293!
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0. Program arguments: tpuc-opt yolov8m_bm1684x_f32_final.mlir --init "--codegen=model_file=yolov8m_fp32_1b.bmodel embed_debug_info=False model_version=latest bmodel_only=False gdma_check=True" --deinit --mlir-print-debuginfo -o /dev/null
#0 0x00005651f4152ff7 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x91cff7)
#1 0x00005651f4150d1e (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x91ad1e)
#2 0x00005651f415397a (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x91d97a)
#3 0x00007fd3ceb85520 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/lib/third_party/libc.so.6+0x42520)
#4 0x00007fd3cebd99fc pthread_kill (/usr/local/lib/python3.10/dist-packages/tpu_mlir/lib/third_party/libc.so.6+0x969fc)
#5 0x00007fd3ceb85476 gsignal (/usr/local/lib/python3.10/dist-packages/tpu_mlir/lib/third_party/libc.so.6+0x42476)
#6 0x00007fd3ceb6b7f3 abort (/usr/local/lib/python3.10/dist-packages/tpu_mlir/lib/third_party/libc.so.6+0x287f3)
#7 0x00005651f4150b41 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x91ab41)
#8 0x00005651f5b8a2c2 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x23542c2)
#9 0x00005651f5b89dfa (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x2353dfa)
#10 0x00005651f5908c9d (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x20d2c9d)
#11 0x00005651f59021f7 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x20cc1f7)
#12 0x00005651f4b95a4e (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x135fa4e)
#13 0x00005651f42a534e (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0xa6f34e)
#14 0x00005651f4b8fb08 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x1359b08)
#15 0x00005651f4b8dc5b (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x1357c5b)
#16 0x00005651f4b8a93e (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x135493e)
#17 0x00005651f5a5db04 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x2227b04)
#18 0x00005651f5a5e131 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x2228131)
#19 0x00005651f5a605d8 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x222a5d8)
#20 0x00005651f414467b (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x90e67b)
#21 0x00005651f4143a44 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x90da44)
#22 0x00005651f5c7b058 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x2445058)
#23 0x00005651f413dd4a (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x907d4a)
#24 0x00005651f413e214 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x908214)
#25 0x00005651f413cc5a (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x906c5a)
#26 0x00007fd3ceb6cd90 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/lib/third_party/libc.so.6+0x29d90)
#27 0x00007fd3ceb6ce40 __libc_start_main (/usr/local/lib/python3.10/dist-packages/tpu_mlir/lib/third_party/libc.so.6+0x29e40)
#28 0x00005651f413c065 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x906065)
Aborted (core dumped)
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/tools/model_deploy.py", line 678, in <module>
tpu_patterns = tool.build_model()
File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/tools/model_deploy.py", line 402, in build_model
patterns = mlir_to_model(
File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/utils/mlir_shell.py", line 1066, in mlir_to_model
_os_system(cmd, log_level=log_level)
File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/utils/mlir_shell.py", line 495, in _os_system
raise RuntimeError("[!Error]: {}".format(cmd_str))
RuntimeError: [!Error]: tpuc-opt yolov8m_bm1684x_f32_final.mlir --codegen="model_file=yolov8m_fp32_1b.bmodel embed_debug_info=False model_version=latest bmodel_only=False gdma_check=True" -o /dev/null
What am I doing wrong, or is there any other way to transpose an image from within the model, without resorting to having the client transpose the image before sending?
Hello.
My original ONNX model was NCHW.
My goal is to get rid of client side image-transposing by making the transpose operation a part of the bmodel itself.
We have a yolov8m based model, and the following snippet has been modified from the official sophon-demo yolov8_plus_det app. It works perfectly, with no issues.
model_transform.py \ --model_name $model_name \ --model_def $onnx_path \ --input_shapes [[$1,3,352,640]] \ --mean 0.0,0.0,0.0 \ --scale 0.0039216,0.0039216,0.0039216 \ --keep_aspect_ratio \ --pixel_format rgb \ --mlir ${model_name}_$1b.mlirThe problem arises when I try to transpose (notice
--input_shapesand--channel_format:model_transform.py \ --model_name $model_name \ --model_def $onnx_path \ --input_shapes [[$1,352,640,3]] \ --channel_format nhwc \ --mean 0.0,0.0,0.0 \ --scale 0.0039216,0.0039216,0.0039216 \ --keep_aspect_ratio \ --pixel_format rgb \ --mlir ${model_name}_$1b.mlirIn this case, the MLIR output gets generated just fine, but the
model_deployscript fails.model_deploy.py \ --mlir ${model_name}_$1b.mlir \ --quantize F32 \ --chip $target \ --model ${model_name}_fp32_$1b.bmodelWhat am I doing wrong, or is there any other way to transpose an image from within the model, without resorting to having the client transpose the image before sending?