Safety
THOM: Generating Physically Plausible Hand-Object Meshes From Text
arXiv:2604.02736v3 Announce Type: replace Abstract: Generating photorealistic 3D hand-object interactions (HOIs) from text is important for applications like robotic grasping and AR/VR content creatio
arXiv:2604.02736v3 Announce Type: replace Abstract: Generating photorealistic 3D hand-object interactions (HOIs) from text is important for applications like robotic grasping and AR/VR content creation. In practice, however, achieving both visual fidelity and physical plausibility remains difficult, as mesh extraction from text-generated Gaussians is inherently ill-posed and the resulting meshes are often unreliable for physics-based optimization. We present THOM, a training-free framework that generates physically plausible 3D HOI meshes directly from text prompts, without requiring template object meshes. THOM follows a two-stage pipeline: it first generates hand and object Gaussians guided by text, and then refines their interaction using physics-based optimization. To enable reliable interaction modeling, we introduce a mesh extraction method with an explicit vertex-to-Gaussian mapping, which enables topology-aware regularization. We further improve physical plausibility through contact-aware optimization and vision-language model (VLM)-guided translation refinement. Extensive experiments show that THOM produces high-quality HOIs with strong text alignment, visual realism, and interaction plausibility.
Related
- MeshOn: Intersection-Free Mesh-to-Mesh Composition
- ViserDex: Visual Sim-to-Real for Robust Dexterous In-hand Reorientation
- A Physical Agentic Loop for Language-Guided Grasping with Execution-State Monitoring
- Physically Plausible Human-Object Rendering from Sparse Views via 3D Gaussian Splatting
Source: arXiv cs.CV | 2026-04-14