Dex-LAMP

Real-world dexterous manipulation

LAMP

Latent Motion Prior-Guided Real-World Learning for Dexterous Hand Manipulation

LAMP learns a compact hand-motion prior from demonstrations, then uses the same decodable latent interface for behavior cloning and online residual reinforcement learning.

Code

Core Idea

Structured exploration via a latent prior.

Raw hand-action spaces make real robot learning brittle. LAMP keeps arm commands in the native action space, while hand commands pass through a demonstration-trained latent prior that stays smooth, compact, and decodable for both imitation learning and residual reinforcement learning.

Latent Prior Module

Compress high-dimensional hand motion into a structured latent space.

The LMPM encoder learns the compact interface from hand-motion histories. During policy learning, the frozen decoder turns latent commands back into executable dexterous hand motion.

LMPM module compressing original high-dimensional hand motion into a low-dimensional latent space and decoding it back into executable hand motion

Pipeline

From demonstrations to autonomous rollouts.

LAMP pipeline: LMPM pretraining learns a latent action space from hand histories, imitation learning trains arm and latent hand policies through the frozen prior, and reinforcement learning adds residual actions for autonomous robot rollouts.
Pretrain

LMPM turns hand histories into a compact latent action space.

Offline demonstrations teach an encoder-decoder prior that maps smooth latent samples back to executable hand motion.

Imitate

The visuo-motor policy learns arm actions and latent hand actions together.

The arm stays in its native command space while the hand passes through the frozen LMPM decoder.

Improve

Online residual learning corrects the policy without leaving the prior.

Residual actions refine both the arm and latent hand commands during real-world autonomous trials.

Results

We evaluate LAMP across four real-robot tasks.

56.25%average imitation success
98.75%average final online RL success
3 / 4tasks reach 100% final success
48real-robot rollout videos
Grasp & Place75% IL to 100% RL
Open Drawer50% IL to 100% RL
Pull Tissue45% IL to 95% RL
Assemble Box55% IL to 100% RL
LAMP success rates after imitation learning (IL) and final online residual RL on real robots.
Action Interface Grasp & Place Open Drawer Pull Tissue Assemble Box
LAMP 75% IL100% RL 50% IL100% RL 45% IL95% RL 55% IL100% RL
w/o low-dimensional bottleneck 40% IL35% RL 65% IL85% RL 15% IL80% RL 5% IL20% RL
w/o history-conditioned encoder 70% IL95% RL 35% IL90% RL 40% IL60% RL 15% IL50% RL
Raw BC 0% IL15% RL 20% IL0% RL 0% IL0% RL 0% IL0% RL

Rollouts

Every task, every group, both IL and RL.

48 videos

Citation

BibTeX

@misc{yang2026lamplatentmotionpriorguided,
      title={LAMP: Latent Motion Prior-Guided Real-World Learning for Dexterous Hand Manipulation},
      author={Xinye Yang and Zhiyuan Ma and Hongze Yu and Yuanpei Chen and Yaodong Yang and Xiaojie Chai and Xinlei Chen and Chao Yu},
      year={2026},
      eprint={2607.06323},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2607.06323},
}