deep-rl-grasping

Deep Reinforcement Learning on Robotics Grasping

Train robotics model with integrated curriculum learning-based gripper environment. Choose from different perception layers depth, RGB-D. Run pretrained models with SAC, BDQ and DQN algorithms. Test trained algorithms in different scenes and domains.

Master’s thesis PDF

Prerequisites (CPU)

Install anaconda. Start a clean conda environment.

conda create -n grasp_env python=3.6
conda activate grasp_env

python manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir trained_models/SAC_full --timestep 100000 -v

Prerequisites (GPU)

conda create -n grasp_env python=3.6
conda activate grasp_env
conda install -c conda-forge cudatoolkit=10.0 cudnn=7.6.5
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/

In setup.py change:

'tensorflow==1.14.0',
to
'tensorflow_gpu==1.14.0',

Installation

Use pip to install the dependencies.

pip install -e .

If using GPU you can check if it was successful with: ´´´ python -c “import tensorflow as tf; print(tf.config.experimental.list_physical_devices(‘GPU’))” ´´´

Run Models

train_stable_baselines script provides the functionality of running and training models.

For running models ‘manipulation_main/training/train_stable_baselines.py’ takes the following arguments

• –model - trained model file e.g trained_models/SAC_full_depth_1mbuffer/best_model/best_model.zip
• -t - use test dataset if not given runs on training dataset
• -v - visualize the model (faster without the -v option)
• -s - run stochastic model if not deterministic

For running functionality run sub-parser needs to be passed to the script.

python manipulation_main/training/train_stable_baselines.py run --model trained_models/SAC_full_depth_1mbuffer/best_model/best_model.zip -v -t

Train models

For training models ‘manipulation_main/training/train_stable_baselines.py’ takes the following arguments

• –config - config file (e.g ‘config/simplified_object_picking.yaml’ or ‘config/gripper_grasp.yaml’)
• –algo - algorithm to use(e.g BDQ, DQN, SAC, TRPO)
• –model_dir - name of the folder to host the trained model logs and best performing model on validation set.
• -sh - use shaped reward function (Only makes sense for Full Environment version)
• -v - visualize the model

For training functionality train sub-parser needs to be passed to the script.

python manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir trained_models/SAC_full --timestep 100000 -v

Running the tests

To run the gripperEnv related test use

pytest tests_gripper

• Domain and Scene Transfer

• Different Perception Layers

• Ablation Studies

  

• Training Environment

  

• Domain transfer performance

  

Authors

• Baris Yazici - Initial work - BarisYazici

Citing the Project

To cite the master’s thesis:

@MastersThesis{Yazici2020,
    author     =     {Yazici Baris},
    title     =     {{Branch Dueling Deep Q-Networks for Robotics Applications}},
    school     =     {Technical University of Munich},
    year     =     {2020},
    howpublished = {\url{https://github.com/BarisYazici/tum_masters_thesis}}
}

License

This project is licensed under the MIT License - see the LICENSE.md file for details

Acknowledgments

• Mahmoud Akl (supervisor)
• Breyer Michel (author of https://arxiv.org/abs/1803.04996)
• https://github.com/atavakol/action-branching-agents