clvrai/furniture clvrai/furniture

  • Stars
    star
    497
  • Rank 88,652 (Top 2 %)
  • Language
    Python
  • License
    MIT License
  • Created about 5 years ago
  • Updated almost 2 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
clvrai/furniture
github

clvrai

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks

IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks

Youngwoon Lee, Edward S. Hu, Joseph J. Lim at USC CLVR lab
[Environment website (https://clvrai.com/furniture)]
[arXiv Paper]
Sawyer Baxter Cursors Jaco Panda

We are announcing the launch of the IKEA Furniture Assembly environment as a first-of-its-kind benchmark for testing and accelerating the automation of physical assembly processes. An agent (Sawyer, Baxter, Cursor) is required to move, align, and connect furniture parts sequentially. The task is completed when all parts are connected.

The IKEA Furniture Assembly environment provides:

  • Comprehensive modeling of furniture assembly task
  • 60+ furniture models
  • Configurable and randomized backgrounds, lighting, textures
  • Realistic robot simulation for Baxter, Sawyer, Jaco, Panda, and more
  • Gym interface for easy RL training
  • Reinforcement learning and imitation learning benchmarks
  • Teleopration with 3D mouse/VR controller

Directories

The structure of the repository:

  • docs: Detail documentation
  • furniture:
    • config: Configuration files for the environments
    • env: Envrionment code of the IKEA furniture assembly environment
    • util: Utility code
    • demo_manual.py: Script for testing the environment with keyboard control
  • furniture-unity: Unity code for the IKEA furniture assembly environment (excluded in this repo due to the size of files, instead download pre-built Unity app)
  • method: Reinforcement learning and imitation learning code (will be updated soon)

(0) Installation

Prerequisites

  • Ubuntu 18.04, MacOS Catalina, Windows10
  • Python 3.7 (pybullet may not work with Python 3.8 or higher)
  • Mujoco 2.0
  • Unity 2018.4.23f1 (Install using Unity Hub)

Installation

git clone https://github.com/clvrai/furniture.git
cd furniture
pip install -e .

See docs/installation.md for more detailed instruction and troubleshooting.
If you are on a headless server, make sure you run a virtual display and use --virtual_display to specify the display number (e.g. :0 or :1).

(1) Human control

You can use WASDQE keys for moving and IJKLUO keys for rotating an end-effector of an agent. SPACE and ENTER are closing and opening the gripper, respectively. C key will connect two aligned parts.

python -m furniture.demo_manual

(2) Gym interface

Gym interface for the IKEA Furniture Assembly environment is also provided. The environment parameters, such as furniture, background, and episode length, can be specified via parameters. (see register functions in furniture/env/__init__.py.

import gym
import furniture

# make an environment
env = gym.make('IKEASawyer-v0', furniture_name="table_lack_0825")

done = False

# reset environment
observation = env.reset()

while not done:
    # simulate environment
    observation, reward, done, info = env.step(env.action_space.sample())

(3) Demonstration generation

We provide the demonstration generation script for 10 furniture models.

python -m furniture.env.furniture_sawyer_gen --furniture_name table_lack_0825 --start_count 0 --n_demos 100

(4) Benchmarking

We provide example commands for table_lack_0825. You can simply change the furniture name to test on other furniture models. For evaluation, you can add --is_train False --num_eval 50 to the training command:

IL Training

BC

python -m run --algo bc --run_prefix bc_table_lack_0825 --env IKEASawyerDense-v0 --furniture_name table_lack_0825 --demo_path demos/Sawyer_table_lack_0825

GAIL

mpirun -np 32 python -m run --algo gail --run_prefix gail_table_lack_0825 --env IKEASawyerDense-v0 --furniture_name table_lack_0825 --demo_path demos/Sawyer_table_lack_0825

GAIL + PPO

mpirun -np 32 python -m run --algo gail --run_prefix gailppo_table_lack_0825 --env IKEASawyerDense-v0 --furniture_name table_lack_0825 --demo_path demos/Sawyer_table_lack_0825 --gail_env_reward 0.5

RL Training

SAC

python -m run --algo sac --run_prefix sac_table_lack_0825 --env IKEASawyerDense-v0 --furniture_name table_dockstra_0279

PPO

mpirun -np 32 python -m run --algo ppo --run_prefix ppo_table_lack_0825 --env IKEASawyerDense-v0 --furniture_name table_dockstra_0279

(5) Documentation

See documentation for installation and configuration details.


(6) References

Our Mujoco environment is developed based on Robosuite and Unity implementation from DoorGym-Unity is used.


(7) Citation

@inproceedings{lee2021ikea,
  title={{IKEA} Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks},
  author={Lee, Youngwoon and Hu, Edward S and Lim, Joseph J},
  booktitle={IEEE International Conference on Robotics and Automation (ICRA)},
  year={2021},
  url={https://clvrai.com/furniture},
}

Contributors

We thank Alex Yin and Zhengyu Yang for their contributions. We would like to thank everyone who has helped IKEA Furniture Assembly Environment in any way.

More Repositories

1

awesome-rl-envs

1,064
star
2

SSGAN-Tensorflow

A Tensorflow implementation of Semi-supervised Learning Generative Adversarial Networks (NIPS 2016: Improved Techniques for Training GANs).
Python
514
star
3

Relation-Network-Tensorflow

Tensorflow implementations of Relational Networks and a VQA dataset named Sort-of-CLEVR proposed by DeepMind.
Python
321
star
4

ACGAN-PyTorch

Python
264
star
5

spirl

Official implementation of "Accelerating Reinforcement Learning with Learned Skill Priors", Pertsch et al., CoRL 2020
Python
186
star
6

furniture-bench

FurnitureBench: Real-World Furniture Assembly Benchmark (RSS 2023)
Python
129
star
7

Representation-Learning-by-Learning-to-Count

A Tensorflow implementation of Representation Learning by Learning to Count
Python
107
star
8

BicycleGAN-Tensorflow

A Tensorflow implementation of BicycleGAN.
Python
101
star
9

Generative-Latent-Optimization-Tensorflow

Tensorflow implementation of Generative Latent Optimization (GLO) proposed by Facebook AI Research
Python
96
star
10

mopa-rl

Motion Planner Augmented Reinforcement Learning for Robot Manipulation in Obstructed Environments (CoRL 2020)
Python
70
star
11

MAML-tf

Tensorflow Implementation of MAML
Python
52
star
12

skimo

Skill-based Model-based Reinforcement Learning (CoRL 2022)
Python
50
star
13

coordination

Learning to Coordinate Manipulation Skills via Skill Behavior Diversification (ICLR 2020)
Python
43
star
14

leaps

Code for Learning to Synthesize Programs as Interpretable and Generalizable Policies in NeurIPS 2021
Python
33
star
15

FeatureControlHRL-Tensorflow

A Tensorflow implementation of Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning
Python
32
star
16

skill-chaining

Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularization (CoRL 2021)
Python
28
star
17

CycleGAN-Tensorflow

A Tensorflow implementation of Cycle-Consistent Adversarial Networks.
Python
26
star
18

new-actions-rl

Jupyter Notebook
23
star
19

goal_prox_il

Generalizable Imitation Learning from Observation via Inferring Goal Proximity (NeurIPS 2021)
Python
22
star
20

skild

Python
21
star
21

i2a-tf

Imagination Augmented Agents in TensorFlow
Python
20
star
22

boss

Code for the paper Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidance, accepted to CoRL 2023 as an Oral Presentation.
Python
18
star
23

create

CREATE Environment for long-horizon physics-puzzle tasks with diverse tools
Python
17
star
24

agile

Official implementation of "Know Your Action Set: Learning Action Relations for Reinforcement Learning", Jain et al., ICLR 2022.
Python
16
star
25

clvr_jaco_play_dataset

Official release of the CLVR Jaco Play Dataset, Dass et al. 2023
Shell
10
star
26

sprint

Code and website for for SPRINT: Scalable Policy Pre-Training via Language Instruction Relabeling
Python
10
star
27

DiscoGAN-Tensorflow

A Tensorflow implementation of DiscoGAN.
Python
8
star
28

tarp

Official implementation of "Task-Induced Representation Learning", Yamada et al., ICLR 2022
Python
8
star
29

idapt

Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Grounding (RSS 2021)
Python
8
star
30

mopa-pd

Distilling Motion Planner Augmented Policies into Visual Control Policies for Robot Manipulation (CoRL 2021)
Python
6
star