Qwen3 VLM: handle scenario specifc latency setting and dataset sample count to submission checker#2613
Qwen3 VLM: handle scenario specifc latency setting and dataset sample count to submission checker#2613zihaok wants to merge 2 commits into
Conversation
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
… in submission checker
9f5e1eb to
d42745e
Compare
|
recheck |
1 similar comment
|
recheck |
|
@pgmpablo157321 @anandhu-eng @tanvi-mlcommons Could you help Zihao to solve the CLA error reporting here. |
Hi @zihaok, have you signed the CLA? |
Zihao is from NVIDIA |
|
Hi @hanyunfan , all the checks now pass |
|
Hi @pgmpablo157321 , could you take a look at this MR? this add support to parse VLM's interactive scenario's log in final submission |
Background:
Qwen3 Offline and Server scenario uses 48289 samples, but Interactive scenario use 8000 samples due to runtime cost.
Hence, the difference between Server and Interactive scenario are
This PR adds this logic in submission checker to address these two scenario differences in Qwen3.