TD3 Applied to the Bittle Robot in Isaac Sim

Attempting to train the Bittle robot dog to walk with reinforcement learning.

• Petoi Bittle URDF Here

• Youtube video to help you Here

Forked TD3 Benchmarking with a Few Files Added:

These files are modified to work with the Isaac Sim environment for the Bittle robot. More information on all of this can be found in the videos that cover my attempts:

There are probably mistakes, errors, redundant, or useless bits of code; I am just sharing where I am so far in case anyone wishes to try their hand. PRs are welcome!

Running the Code:

To actually run, you need to use the Omni version of Python. You can either do this via shebang at the top or by placing my 2 edited files in the Omniverse directory. For example, my files are located here:

/home/h/.local/share/ov/pkg/isaac_sim-2021.1.1/TD3-Bittle-9/

Key Files:

1. TD3-Bittle-16-1.py: This is the main training file.
2. TD3_4.py: This is the modified version of the TD3 model. It’s a larger model.
3. Models directory: Contains the best TD3 model so far. You can run it or attempt to train from here.
4. 20-Bittles-very-long-6.usd: A version of a scene used to play the model. Feel free to create your own from here in the sim. Should be located in the Isaac directory of the Omniverse directory.
5. 20-Bittles-very-long.usd: Used mostly for training to prevent Bittles from running into each other. There’s a way to make them clip each other, but I don't know it at the moment. Should be located in the Isaac directory of the Omniverse directory.
6. Play-TD3-Bittle-MANY.py: Used to play models, essentially just keeps exploiting.
7. graph_reward.py: Used to graph the rewards of the exploration and exploiting agents live while training.