Add antialias support to nn.Upsample for linear and cubic modes#3677
Add antialias support to nn.Upsample for linear and cubic modes#3677lyonsno wants to merge 1 commit into
Conversation
c1f90b8 to
82a1799
Compare
Pablosinyores
left a comment
There was a problem hiding this comment.
I spent some time verifying this one against the pure-Python layer (built a small harness with the prebuilt wheel's core + this branch's upsample.py), and the antialias math checks out. A few things I confirmed independently:
-
Kernels are correct.
_triangle_kernelgives 1 / 0.5 / 0 at x = 0 / 0.5 / 1._cubic_kernel(Keys, a=-0.5) gives 1 at x=0, 0 at x=1 and x=2, and 0.5625 at x=0.5 — matches the closed form((a+2)x - (a+3))x² + 1for a=-0.5. Switching the coefficient from -0.75 (OpenCV) to -0.5 (PIL) for the antialiased path is exactly what PyTorch does, and the comment update in_cubic_indiceskeeps the two paths self-documenting. -
The filter weights are right. For a 1-D linear downscale at
scale=0.5, N=4, align_corners=False, an impulse test gave per-tap weights[0.4286, 0.4286, 0.1429]onx[0..2]for the first output pixel — which is what you get by hand from a triangle filter widened to support1/scale = 2and renormalized (0.75, 0.75, 0.25/ 1.75). So thesupport = kernel_radius / scale, zero-OOB, renormalize sequence is doing the right thing. -
Tap coverage is safe.
num_taps = ceil(support) + 1with the symmetricfloor(center) + krange always spans every in-support integer tap; the extras fall outside the kernel and get exactly 0 weight, so nothing in-support is dropped. -
Multi-axis broadcasting is fine. I briefly worried
expand_dims(w, -1)would misalign on the first axis of a 2-D resize, but_scaled_indicesalready reshapes indices toshape[dim] = -1, so the weight ends up(out_i, 1, 1)and broadcasts along the correct axis. A non-square 2-D downscale(1,8,6,2) -> (1,4,3,2)runs and produces the right shape.
Two small things worth considering, neither blocking:
-
Test coverage is square + integer-ratio only. All the
test_torch_upsample_antialiascases use square inputs with equal per-axis scales, so the non-square path and per-axis-different scales aren't exercised against PyTorch even though they run. Since the note already flags non-integer scales as potentially divergent (#2186), it might be worth at least one non-square integer-ratio case (e.g.(8,4) -> (4,2)) to lock in the per-axis behavior. -
antialias=True still routes upscale through the separable path. For
scale >= 1the triangle case reduces to plain linear (3 taps, the far one gets 0 weight), so it's correct, just slightly more work than the non-AA path. Not important, but if you wanted,antialiascould no-op when every scale factor is>= 1.
Nice, clean addition overall — the separable rewrite and the OOB-zero-then-renormalize approach are the standard way to do this.
|
Thanks for the independent harness and the detailed check. The tap-coverage and per-axis broadcasting confirmations are especially helpful. I agree a non-square integer-ratio case is worth adding to lock down that path; I’ll add one. I’m inclined to leave the upscale antialias=True fast-path optimization out of this PR unless maintainers want it, since the current route preserves correctness and keeps the patch focused. Appreciate you taking the time to verify it independently. |
Summary
Adds an
antialiasparameter tonn.Upsample,upsample_linear(), andupsample_cubic()for the defaultalign_corners=Falseinterpolation path,matching PyTorch's
F.interpolate(antialias=True)on the tested bilinear andbicubic cases.
When downsampling with
antialias=True, the interpolation kernel widensproportionally to the inverse scale factor, applying a low-pass filter that
prevents aliasing artifacts. For
"cubic"mode, this also switches thekernel coefficient from
a=-0.75(OpenCV) toa=-0.5(PIL), matchingPyTorch's convention.
Motivation: PyTorch-origin vision model ports that use
F.interpolate(antialias=True)currently hit a parity gap. Representativeexamples include DINOv2, MoGe, SDXL VAE, and many modern vision architectures.
Implementation
_aa_indices()function that takes a kernel callable and supportradius, eliminating duplication between linear and cubic paths.
1/scalein input pixels.a=-0.5(PIL convention).mx.takepath so very smalldownscales do not construct a cartesian product of per-axis taps. No
C++/Metal changes.
antialias=Truewithalign_corners=TrueraisesValueErrorin bothUpsampleand the direct linear/cubic helpers; that coordinate convention isdeliberately outside this patch's supported surface.
antialias=Truewith"nearest"mode raisesValueErrorat init time.Parity
Tested against PyTorch
F.interpolate(antialias=True)for 2D bilinear andbicubic
align_corners=Falsepaths at multiple integer-ratio scale factors andinput sizes:
Cubic upscale with
antialias=Truealso matches PyTorch (thea=-0.5coefficient applies to both up and downscale when antialias is enabled).
Tests
11 test methods, 162 subtests:
test_torch_upsample_antialias: bilinear + bicubic at 0.5x/0.25x,sizes 4×4 to 64×64, square/non-square, asymmetric scales per dimension.
All cross-validated against PyTorch.
test_antialias_upscale_linear_is_noop: linear antialias has noeffect when upscaling.
test_antialias_upscale_cubic_matches_pytorch: cubic antialiasmatches PyTorch on upscale (a=-0.5 coefficient change).
test_antialias_non_integer_scale_smoke: property tests fornon-integer scales (shape, finite output; linear AA boundedness).
test_antialias_1d_smoke: 1D spatial input smoke test (PyTorchdoesn't support 1D antialias).
test_antialias_uses_separable_path: verifies antialiased linear andcubic paths do not call the cartesian
productinterpolation path.test_antialias_align_corners_raises: ValueError at init for downscaleand upscale.
test_antialias_align_corners_direct_functions_raise:direct
upsample_linear()andupsample_cubic()helpers reject the sameunsupported contract for downscale and upscale.
test_antialias_nearest_raises: ValueError at init.test_antialias_differs_from_non_antialias: AA output has lowervariance than non-AA on random data.
Focused existing upsample coverage also passes:
python/tests/test_nn.py -q -k upsample.Limitations
antialias=Truewithalign_corners=Trueis not supported and raisesValueError.align_corners=False(the default) works correctly.per-axis tap expansion, but very small downscales still require more
per-axis taps than non-AA interpolation. A fused C++/Metal kernel could
improve both AA and non-AA paths in future work.
a pre-existing difference in
_scaled_indicesstep computation.