Mesh-based Non-collision Constraints#10
Conversation
There was a problem hiding this comment.
🤖 Isaac Lab-Arena Review Bot
Summary
This PR adds an opt-in CollisionMode.MESH to the relation solver/placer: a greedy sphere decomposition of each object's mesh plus a differentiable Warp SDF kernel, used both in the optimisation loss (NoCollisionLossStrategy) and the post-solve validator (_validate_no_overlap_mesh). The implementation is careful — warp imports are deferred, the new code stays in the placement domain (correct layer), MESH defaults off (BBOX stays the default path), and the test suite is thorough. The main thing worth resolving is whether the USD-mesh entry point is meant to ship in this PR at all.
Design, Boundaries & Scope
Layering looks right. Mesh collision is pure geometric computation, so it legitimately belongs in relations/ alongside the rest of the solver — it doesn't take a live env, step the sim, or read sim state. The one lean-dependency concern (the invariant calls out warp explicitly) is handled correctly: warp_mesh_manager/warp_sdf_kernels are only ever lazy-imported, so importing object_placer in BBOX mode never drags in warp. Default behaviour is unchanged (collision_mode=CollisionMode.BBOX), so blast radius is contained.
Scope question — is the feature actually wired to real objects? ObjectBase.get_collision_mesh() returns None and the only override in the repo is DummyObject (which is handed a mesh directly). No Object (the USD-backed production type) overrides it, and the new extract_trimesh_from_usd() helper — the obvious bridge from a USD asset to a trimesh — has no caller anywhere. The net effect is that for every real (USD) object, MESH mode silently falls back to AABB; the new path is only exercisable through DummyObject in the tests/benchmark. That's fine if USD wiring is a deliberate follow-up, but then extract_trimesh_from_usd is speculative code that's better deferred to the MR that uses it (it's untested and unreachable as shipped). If it is meant to work here, an Object.get_collision_mesh() override calling the helper seems to be the missing piece. Could you clarify the intent? (Flagged inline.)
Findings
🟡 Warning: isaaclab_arena/utils/usd_helpers.py:208 — extract_trimesh_from_usd has no caller and no Object wires it into get_collision_mesh(); as shipped MESH mode never engages for production objects. Wire it or defer to the follow-up MR. (inline)
🔵 Improvement: isaaclab_arena/utils/usd_helpers.py:224 — Raises: docstring section; Arena convention is to omit Raises:. (inline, with suggestion)
🔵 Improvement: isaaclab_arena/relations/object_placer.py:673 — the validator treats the SDF sentinel as collision-free, where the loss path (sphere_penetration_loss) explicitly warns about it. (inline)
🔵 Improvement: warp_sdf_kernels.py:902 / object_placer.py:609 — two small state smells: sphere_penetration_loss._warned_sentinel stashes a one-shot flag on the function object, and _get_cpu_mesh_manager lazily creates self._cpu_mesh_manager via hasattr rather than initialising it in __init__. Both work, but initialising the cache attribute in __init__ (set to None) keeps the object's state space explicit. Minor.
🔵 Improvement: relation_loss_strategies.py:480 — _compute_mesh_loss loops over the batch in Python (for b in range(batch_size)), launching Warp kernels per env. Fine for the opt-in/known-slow MESH path, but worth a comment noting it's the per-env-loop cost if multi-env placement ever routes through here.
Test Coverage
Strong for a pure-compute feature: unit coverage of sphere decomposition, cache identity, dispatch routing (BBOX vs MESH vs no-mesh fallback), the autograd backward gradient, the rotated-anchor and yaw guards, plus end-to-end solver/placer separation tests. These are non-sim (pure trimesh/warp), so the inner/outer run_simulation_app_function pattern correctly doesn't apply, and @requires_warp gates the warp-dependent ones. Two gaps worth considering: (1) no test for extract_trimesh_from_usd (related to the wiring question above); (2) test_dispatch_routes_to_mesh_in_mesh_mode asserts only that mesh ≠ AABB loss — a value-level assertion would be more robust, but the dedicated test_mesh_* cases cover the numerics.
Verdict
Minor fixes needed — the implementation is sound and well-tested; the main ask is clarifying whether the unused USD helper / real-object wiring belongs in this PR, plus the docstring nit.
| ) | ||
|
|
||
|
|
||
| def extract_trimesh_from_usd( |
There was a problem hiding this comment.
🟡 Not wired in anywhere. This helper has no caller in the PR (or repo), and no Object overrides ObjectBase.get_collision_mesh() to use it — only DummyObject returns a mesh, and it's handed one directly. So for every real USD-backed object, CollisionMode.MESH silently falls back to AABB and this conversion never runs.
Is the USD→mesh path intended for this PR? If so, an Object.get_collision_mesh() that calls extract_trimesh_from_usd(self.usd_path, self.scale) (cached) looks like the missing link. If it's a follow-up, consider deferring this helper to the MR that actually uses it — as shipped it's untested and unreachable.
(Minor: the docstring says "local frame of the default prim" but the code applies ComputeLocalToWorldTransform, i.e. the stage's world frame — worth reconciling.)
| Returns: | ||
| Combined trimesh in the USD default prim's local frame. | ||
|
|
||
| Raises: | ||
| ValueError: If the file cannot be opened or contains no mesh prims. | ||
| """ |
There was a problem hiding this comment.
🔵 Arena's docstring convention (AGENTS.md / PR isaac-sim#747) deliberately omits Raises: sections — suggest dropping it.
| Returns: | |
| Combined trimesh in the USD default prim's local frame. | |
| Raises: | |
| ValueError: If the file cannot be opened or contains no mesh prims. | |
| """ | |
| Returns: | |
| Combined trimesh in the USD default prim's local frame. | |
| """ |
| spheres_a = manager.get_query_spheres(a_mesh, obj=a) | ||
| warp_b = manager.get_warp_mesh(b_mesh, obj=b) | ||
| centers_a_in_b = spheres_a[:, :3] + a_pos - b_pos | ||
| if (mesh_sdf(centers_a_in_b, warp_b) < spheres_a[:, 3] + tolerance).any(): |
There was a problem hiding this comment.
🔵 mesh_sdf returns the 1e6 sentinel when a query finds no mesh face, and here that reads as 1e6 < radius + tolerance → False → "no overlap", silently passing validation. The loss-path twin (sphere_penetration_loss) explicitly warns on this exact case (_SDF_SENTINEL), so the two paths are inconsistent. In practice it's near-unreachable for a valid non-empty mesh (max_dist is 1e6), so this is low-severity — but since this is the accept/reject gate for a placement, mirroring the loss path's sentinel check (or asserting the SDF is finite) would close the gap. Worth a NOTE: either way.
Replay of upstream isaac-sim#771 onto the fork for review. Genuine human-authored diff (12 files, +1289/-33); base/head pushed verbatim so the diff matches the original.