Skip to content

Mesh-based Non-collision Constraints#10

Open
cvolkcvolk wants to merge 3 commits into
replay/pr-771-basefrom
replay/pr-771-mesh-collision
Open

Mesh-based Non-collision Constraints#10
cvolkcvolk wants to merge 3 commits into
replay/pr-771-basefrom
replay/pr-771-mesh-collision

Conversation

@cvolkcvolk

Copy link
Copy Markdown
Owner

Replay of upstream isaac-sim#771 onto the fork for review. Genuine human-authored diff (12 files, +1289/-33); base/head pushed verbatim so the diff matches the original.

@arena-review-bot arena-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Isaac Lab-Arena Review Bot

Summary

This PR adds an opt-in CollisionMode.MESH to the relation solver/placer: a greedy sphere decomposition of each object's mesh plus a differentiable Warp SDF kernel, used both in the optimisation loss (NoCollisionLossStrategy) and the post-solve validator (_validate_no_overlap_mesh). The implementation is careful — warp imports are deferred, the new code stays in the placement domain (correct layer), MESH defaults off (BBOX stays the default path), and the test suite is thorough. The main thing worth resolving is whether the USD-mesh entry point is meant to ship in this PR at all.

Design, Boundaries & Scope

Layering looks right. Mesh collision is pure geometric computation, so it legitimately belongs in relations/ alongside the rest of the solver — it doesn't take a live env, step the sim, or read sim state. The one lean-dependency concern (the invariant calls out warp explicitly) is handled correctly: warp_mesh_manager/warp_sdf_kernels are only ever lazy-imported, so importing object_placer in BBOX mode never drags in warp. Default behaviour is unchanged (collision_mode=CollisionMode.BBOX), so blast radius is contained.

Scope question — is the feature actually wired to real objects? ObjectBase.get_collision_mesh() returns None and the only override in the repo is DummyObject (which is handed a mesh directly). No Object (the USD-backed production type) overrides it, and the new extract_trimesh_from_usd() helper — the obvious bridge from a USD asset to a trimesh — has no caller anywhere. The net effect is that for every real (USD) object, MESH mode silently falls back to AABB; the new path is only exercisable through DummyObject in the tests/benchmark. That's fine if USD wiring is a deliberate follow-up, but then extract_trimesh_from_usd is speculative code that's better deferred to the MR that uses it (it's untested and unreachable as shipped). If it is meant to work here, an Object.get_collision_mesh() override calling the helper seems to be the missing piece. Could you clarify the intent? (Flagged inline.)

Findings

🟡 Warning: isaaclab_arena/utils/usd_helpers.py:208extract_trimesh_from_usd has no caller and no Object wires it into get_collision_mesh(); as shipped MESH mode never engages for production objects. Wire it or defer to the follow-up MR. (inline)

🔵 Improvement: isaaclab_arena/utils/usd_helpers.py:224Raises: docstring section; Arena convention is to omit Raises:. (inline, with suggestion)

🔵 Improvement: isaaclab_arena/relations/object_placer.py:673 — the validator treats the SDF sentinel as collision-free, where the loss path (sphere_penetration_loss) explicitly warns about it. (inline)

🔵 Improvement: warp_sdf_kernels.py:902 / object_placer.py:609 — two small state smells: sphere_penetration_loss._warned_sentinel stashes a one-shot flag on the function object, and _get_cpu_mesh_manager lazily creates self._cpu_mesh_manager via hasattr rather than initialising it in __init__. Both work, but initialising the cache attribute in __init__ (set to None) keeps the object's state space explicit. Minor.

🔵 Improvement: relation_loss_strategies.py:480_compute_mesh_loss loops over the batch in Python (for b in range(batch_size)), launching Warp kernels per env. Fine for the opt-in/known-slow MESH path, but worth a comment noting it's the per-env-loop cost if multi-env placement ever routes through here.

Test Coverage

Strong for a pure-compute feature: unit coverage of sphere decomposition, cache identity, dispatch routing (BBOX vs MESH vs no-mesh fallback), the autograd backward gradient, the rotated-anchor and yaw guards, plus end-to-end solver/placer separation tests. These are non-sim (pure trimesh/warp), so the inner/outer run_simulation_app_function pattern correctly doesn't apply, and @requires_warp gates the warp-dependent ones. Two gaps worth considering: (1) no test for extract_trimesh_from_usd (related to the wiring question above); (2) test_dispatch_routes_to_mesh_in_mesh_mode asserts only that mesh ≠ AABB loss — a value-level assertion would be more robust, but the dedicated test_mesh_* cases cover the numerics.

Verdict

Minor fixes needed — the implementation is sound and well-tested; the main ask is clarifying whether the unused USD helper / real-object wiring belongs in this PR, plus the docstring nit.

)


def extract_trimesh_from_usd(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Not wired in anywhere. This helper has no caller in the PR (or repo), and no Object overrides ObjectBase.get_collision_mesh() to use it — only DummyObject returns a mesh, and it's handed one directly. So for every real USD-backed object, CollisionMode.MESH silently falls back to AABB and this conversion never runs.

Is the USD→mesh path intended for this PR? If so, an Object.get_collision_mesh() that calls extract_trimesh_from_usd(self.usd_path, self.scale) (cached) looks like the missing link. If it's a follow-up, consider deferring this helper to the MR that actually uses it — as shipped it's untested and unreachable.

(Minor: the docstring says "local frame of the default prim" but the code applies ComputeLocalToWorldTransform, i.e. the stage's world frame — worth reconciling.)

Comment on lines +221 to +226
Returns:
Combined trimesh in the USD default prim's local frame.

Raises:
ValueError: If the file cannot be opened or contains no mesh prims.
"""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Arena's docstring convention (AGENTS.md / PR isaac-sim#747) deliberately omits Raises: sections — suggest dropping it.

Suggested change
Returns:
Combined trimesh in the USD default prim's local frame.
Raises:
ValueError: If the file cannot be opened or contains no mesh prims.
"""
Returns:
Combined trimesh in the USD default prim's local frame.
"""

spheres_a = manager.get_query_spheres(a_mesh, obj=a)
warp_b = manager.get_warp_mesh(b_mesh, obj=b)
centers_a_in_b = spheres_a[:, :3] + a_pos - b_pos
if (mesh_sdf(centers_a_in_b, warp_b) < spheres_a[:, 3] + tolerance).any():

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 mesh_sdf returns the 1e6 sentinel when a query finds no mesh face, and here that reads as 1e6 < radius + toleranceFalse → "no overlap", silently passing validation. The loss-path twin (sphere_penetration_loss) explicitly warns on this exact case (_SDF_SENTINEL), so the two paths are inconsistent. In practice it's near-unreachable for a valid non-empty mesh (max_dist is 1e6), so this is low-severity — but since this is the accept/reject gate for a placement, mirroring the loss path's sentinel check (or asserting the SDF is finite) would close the gap. Worth a NOTE: either way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants