Skip to content

Transposing via --channel_format and --input_shapes fails #267

Description

@masroof-maindak

Hello.

My original ONNX model was NCHW.

My goal is to get rid of client side image-transposing by making the transpose operation a part of the bmodel itself.

We have a yolov8m based model, and the following snippet has been modified from the official sophon-demo yolov8_plus_det app. It works perfectly, with no issues.

model_transform.py \
        --model_name $model_name \
        --model_def $onnx_path \
        --input_shapes [[$1,3,352,640]] \
        --mean 0.0,0.0,0.0 \
        --scale 0.0039216,0.0039216,0.0039216 \
        --keep_aspect_ratio \
        --pixel_format rgb \
        --mlir ${model_name}_$1b.mlir

The problem arises when I try to transpose (notice --input_shapes and --channel_format:

    model_transform.py \
        --model_name $model_name \
        --model_def $onnx_path \
        --input_shapes [[$1,352,640,3]] \
        --channel_format nhwc \
        --mean 0.0,0.0,0.0 \
        --scale 0.0039216,0.0039216,0.0039216 \
        --keep_aspect_ratio \
        --pixel_format rgb \
        --mlir ${model_name}_$1b.mlir

In this case, the MLIR output gets generated just fine, but the model_deploy script fails.

model_deploy.py \
        --mlir ${model_name}_$1b.mlir \
        --quantize F32 \
        --chip $target \
        --model ${model_name}_fp32_$1b.bmodel
[Running]: tpuc-opt yolov8m_bm1684x_f32_final.mlir --codegen="model_file=yolov8m_fp32_1b.bmodel embed_debug_info=False model_version=latest bmodel_only=False gdma_check=True" -o /dev/null
The dir path of compiler_profile is "./"
bmcpu init: skip cpu_user_defined
Cannot open libusercpu.so, disable user cpu layer.
size does not match for tensor model.0.conv.weight
readTensor failed
UNREACHABLE executed at /__w/tpu-mlir/tpu-mlir/lib/Support/TensorFile.cpp:293!
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: tpuc-opt yolov8m_bm1684x_f32_final.mlir --init "--codegen=model_file=yolov8m_fp32_1b.bmodel embed_debug_info=False model_version=latest bmodel_only=False gdma_check=True" --deinit --mlir-print-debuginfo -o /dev/null
 #0 0x00005651f4152ff7 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x91cff7)
 #1 0x00005651f4150d1e (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x91ad1e)
 #2 0x00005651f415397a (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x91d97a)
 #3 0x00007fd3ceb85520 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/lib/third_party/libc.so.6+0x42520)
 #4 0x00007fd3cebd99fc pthread_kill (/usr/local/lib/python3.10/dist-packages/tpu_mlir/lib/third_party/libc.so.6+0x969fc)
 #5 0x00007fd3ceb85476 gsignal (/usr/local/lib/python3.10/dist-packages/tpu_mlir/lib/third_party/libc.so.6+0x42476)
 #6 0x00007fd3ceb6b7f3 abort (/usr/local/lib/python3.10/dist-packages/tpu_mlir/lib/third_party/libc.so.6+0x287f3)
 #7 0x00005651f4150b41 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x91ab41)
 #8 0x00005651f5b8a2c2 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x23542c2)
 #9 0x00005651f5b89dfa (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x2353dfa)
#10 0x00005651f5908c9d (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x20d2c9d)
#11 0x00005651f59021f7 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x20cc1f7)
#12 0x00005651f4b95a4e (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x135fa4e)
#13 0x00005651f42a534e (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0xa6f34e)
#14 0x00005651f4b8fb08 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x1359b08)
#15 0x00005651f4b8dc5b (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x1357c5b)
#16 0x00005651f4b8a93e (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x135493e)
#17 0x00005651f5a5db04 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x2227b04)
#18 0x00005651f5a5e131 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x2228131)
#19 0x00005651f5a605d8 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x222a5d8)
#20 0x00005651f414467b (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x90e67b)
#21 0x00005651f4143a44 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x90da44)
#22 0x00005651f5c7b058 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x2445058)
#23 0x00005651f413dd4a (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x907d4a)
#24 0x00005651f413e214 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x908214)
#25 0x00005651f413cc5a (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x906c5a)
#26 0x00007fd3ceb6cd90 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/lib/third_party/libc.so.6+0x29d90)
#27 0x00007fd3ceb6ce40 __libc_start_main (/usr/local/lib/python3.10/dist-packages/tpu_mlir/lib/third_party/libc.so.6+0x29e40)
#28 0x00005651f413c065 (/usr/local/lib/python3.10/dist-packages/tpu_mlir/bin/tpuc-opt+0x906065)
Aborted (core dumped)
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/tools/model_deploy.py", line 678, in <module>
    tpu_patterns = tool.build_model()
  File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/tools/model_deploy.py", line 402, in build_model
    patterns = mlir_to_model(
  File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/utils/mlir_shell.py", line 1066, in mlir_to_model
    _os_system(cmd, log_level=log_level)
  File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/utils/mlir_shell.py", line 495, in _os_system
    raise RuntimeError("[!Error]: {}".format(cmd_str))
RuntimeError: [!Error]: tpuc-opt yolov8m_bm1684x_f32_final.mlir --codegen="model_file=yolov8m_fp32_1b.bmodel embed_debug_info=False model_version=latest bmodel_only=False gdma_check=True" -o /dev/null

What am I doing wrong, or is there any other way to transpose an image from within the model, without resorting to having the client transpose the image before sending?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions