PhysHanDI: Physics-Based Reconstruction of Hand-Deformable Object Interactions

Researchers have introduced PhysHanDI, a novel framework designed for the full 3D reconstruction of interactions between human hands and non-rigid, deformable objects. Existing methods often struggle with highly flexible materials like cloth or stuffed animals, typically focusing on rigid objects or lacking complete hand reconstruction. PhysHanDI addresses this limitation by physically simulating object deformations driven by forces from densely reconstructed 3D hand motions. This approach ensures that the resulting object dynamics are both physically plausible and coherent with hand movements. Additionally, the framework utilizes inverse physics to refine and improve hand reconstruction based on the simulated object deformations. Experimental results demonstrate that PhysHanDI outperforms current state-of-the-art baselines in both reconstruction accuracy and future prediction capabilities. This advancement significantly bridges the gap in modeling real-world hand-object interactions involving soft, deformable materials, offering potential applications in robotics, virtual reality, and computer vision.