Real-Time Cloth State Estimation (Ongoing)

Cloth State Estimation Using EKF

This project focuses on estimating the state of deformable cloth using an Extended Kalman Filter (EKF). Estimating the state of cloth is critical for robotic manipulation tasks involving fabrics, enabling precise control and interaction with deformable objects. The system combines a physics-based prediction model with real-time observations from AprilTags to achieve accurate state estimation.

Objectives

• Develop a robust EKF-based framework for tracking the state of cloth.
• Integrate a physics prediction model (CIPC) for cloth dynamics.
• Utilize AprilTags for real-time observations to refine the state estimation.
• Minimize latency to meet real-time requirements.

Implementation Overview

• Observation Model
  • AprilTags are used to gather the real-time positions of markers attached to the cloth.
  • Observations are processed to correct the predicted state of the cloth.
• Prediction Model
  • A physics-based model, CIPC (Compliant Implicit Physics Constraints), predicts the cloth’s state based on dynamics and applied forces.
  • EKF incorporates these predictions to maintain a probabilistic representation of the cloth’s state.

Key Features

• Low Latency: The EKF implementation achieves a processing latency of ~ 0.7 seconds, enabling real-time update
• Simulation to Reality: Incorporates physical constraints to bridge the gap between simulation and real-world manipulation
• Scalable Framework: Can be extended for applications beyond cloth manipulation, such as flexible or articulated objects

Results

• Accuracy: Demonstrated significant improvement in tracking precision compared to standard filtering techniques.
• Robustness: The system performs reliably even with partial or noisy observations.

Video Demonstration