Skip to content

Add fp16 mixed precision softmax - #1950

Draft
kaix-nv wants to merge 28 commits into
mainfrom
kaix/sparse_attn_quant_mixed_fp16
Draft

Add fp16 mixed precision softmax#1950
kaix-nv wants to merge 28 commits into
mainfrom
kaix/sparse_attn_quant_mixed_fp16

Document mixed-FP16 attention serving

a3f54ac
Select commit
Loading
Failed to load commit list.
Codecov / codecov/project succeeded Jul 9, 2026 in 1s

67.95% (+6.73%) compared to 75b5803

View this Pull Request on Codecov

67.95% (+6.73%) compared to 75b5803

Details

Codecov Report

❌ Patch coverage is 54.81799% with 211 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.95%. Comparing base (75b5803) to head (a3f54ac).
⚠️ Report is 20 commits behind head on main.

Files with missing lines Patch % Lines
...torch/kernels/common/attention/decode_attention.py 13.11% 106 Missing ⚠️
...t/torch/kernels/quantization/attention/bmm2_qdq.py 0.00% 47 Missing ⚠️
...delopt/torch/kernels/common/attention/triton_fa.py 37.70% 38 Missing ⚠️
.../torch/sparsity/attention_sparsity/plugins/vllm.py 93.13% 14 Missing ⚠️
...t/torch/kernels/quantization/common/nvfp4_quant.py 0.00% 4 Missing ⚠️
...ernels/quantization/attention/softmax_fakequant.py 90.00% 1 Missing ⚠️
modelopt/torch/quantization/plugins/vllm.py 94.73% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1950      +/-   ##
==========================================
+ Coverage   61.21%   67.95%   +6.73%     
==========================================
  Files         515      523       +8     
  Lines       57245    62047    +4802     
==========================================
+ Hits        35043    42164    +7121     
+ Misses      22202    19883    -2319

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
View more details on Codecov