LmDeploy showcase great performance on inference on Cuda and it could be great to wrap a node called dora-lmdeploy that could specify the model we want to use and handle the logic around receiving text and images.
It would probably be very similar to https://github.com/dora-rs/dora/blob/main/node-hub/dora-qwen2-5-vl except that we replace transformers with lmdeploy pipelines.
LmDeploy showcase great performance on inference on Cuda and it could be great to wrap a node called dora-lmdeploy that could specify the model we want to use and handle the logic around receiving text and images.
It would probably be very similar to https://github.com/dora-rs/dora/blob/main/node-hub/dora-qwen2-5-vl except that we replace transformers with lmdeploy pipelines.