Tutorial for simulating Bittle In Isaac Sim

By following the instruction, you will be able to simulate some pre-defined gestures or motions for Bittle. This video(https://www.youtube.com/watch?v=phTnbmXM06g) from sentdex is my main reference and it is really helpful. You could buy a real Bittle from https://www.petoi.com/collections/robots/products/petoi-bittle-robot-dog?variant=40388667834552.

I. Reference Link&Materials

sentdex’s video: https://www.youtube.com/watch?v=phTnbmXM06g

The actual Petoi Bittle robot can be found here:

https://www.petoi.com/collections/robots/products/petoi-bittle-robot-dog?variant=40388667834552

The documentation for Petoi Bittle robot can be found here:

https://bittle.petoi.com/11-tutorial-on-creating-new-skills

Nvidia Omniverse: https://www.nvidia.com/en-us/omniverse/

Nvidia Issac Sim documentation:

https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/requirements.html

Petoi Bittle URDF: https://github.com/AIWintermuteAI/Bittle_URDF

Pre-defined Bittle Command(From Petoi): instinctBittle.h

II. Simulator Preparation

This part introduces how to install Isaac Sim.

Special Notes

Issac Sim only works on the Linux system. My recommendation is to use a computer with a Linux system rather than using a Virtual Machine. From my personal experience, some connection problems may occur when Issac Sim tries to use the graphics card in a Virtual Machine.

Steps

1. Download Nvidia Omniverse from https://www.nvidia.com/en-us/omniverse/.

2. Install Isaac Sim and Cache from Omniverse→Exchange. After installation, Isaac Sim and Cache should show up in Omniverse→Library.

3. Register Nucleus from Omniverse.

3. Check if you could successfully launch Issac Sim. If you fail, you may see some errors or warnings about your drivers. You could take a look at the documentation from Nvidia which is really helpful https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/requirements.html. If that doesn’t work and you are not so familiar with drivers, you may want to pay extra attention to it. I have destroyed my system by changing the driver. My recommendation is to remove the old version and download a new version rather than try to update the old version.

III. Building Your “World”

This part introduces how to build a simple “world” in Isaac Sim which includes a ground plane and Bittle.

Steps

1. Download Bittle URDF from https://github.com/AIWintermuteAI/Bittle_URDF. This folder includes the model of Bittle and URDF is a general method to describe robots. After you extract the folder you will see the following files.

The path used in the “bittle.urdf” is “filename= “../meshed/obj/base_frame.001.obj””. This path defines how the URDF file finds the corresponding geometry file for this part of the robot.

Based on this information, we need to create two extra folders in this “Bittle_URDF-main” folder which are “urdf” and “meshes”.

Then we could drag “bittle.urdf” into the “urdf” folder and “obj” folder into the “meshes” folder as follows.

2. Import Bittle. After you successfully launch Issac Sim, you should see the following interface and then click “import”.

Click “Select and Import” and select the “bittle.urdf” from your computer.

We need to modify the “Fix Base Link” block. Since our final goal is to make Bittle move, we will not want Bittle to be fixed.

Your interface should look like the following figure after all modifications. If you click “play”, you could see Bittle falling down.

3. Build the ground plane. So far, Bittle is in the air and we need to build the ground for our world. You could create a ground plane by clicking “Create→Physics→Ground Plane”. Then you need to drag the ground plane to make it lower than Bittle.

In the end, your “world” should look like the following figure. If you click “play” again, you could see Bittle falling to the ground.

You could also manually control all the joints of Bittle by first selecting the joint in the stage and then changing its target position. “Damping” and “Stiffness” are two parameters related to the drive motor at the joint. We could roughly set the damping to 0 and the stiffness to be 1300 for all the joints. From my understanding, these two values work similarly to Kp gain and Kd gain in PID control. They could be tuned further to improve the moving performance of Bittle.

4. Save the “Bittle.usd” file by clicking “File→save”. (You could give it another name if you want.)

IV. Building the Simulator&Controller

This part will introduce how to send target positions and actuate joints of Bittle from python files rather than change it manually. I make the simulator and the controller separated. The controller will send the target position for each joint to the simulator and the simulator will try to actuate the joint to its corresponding target position. You could also write the controller and the simulator in the same file. The purpose to make them separated is that your controller could be easily implemented to the real Bittle if you want to design some new motions.

4.1 Develop the Simulator

The simulator will be developed based on some sample files from the folder “simple” whose path is “/.local/share/ov/pkg/isaac_sim/python_samples/simple”. Noting that “.local” is a hidden folder.

Steps

1. Copy and rename the file “franka_articulation.py” as “Bittle_articulation.py”;

2. In line31, change the path to your “Bittle.usd” file. Noting that you also need to delete “nucleus_server” if the file is in your local computer and you don’t upload it to the Nucleus;

3. In lines 41 and 42, change “/panda” to “/bittle”. From my understanding, line 41 will try to get all the joints that you want to control by searching the keyword “/bittle”;

4. Create a variable and store the names of all the joints in this variable. We will pick the joint that we want to control from this variable in the next step. We totally have 8 controllable joints in Bittle.

5. Change line 63 to line 69 to the following code. Function “find_articulation_dof()” will find the corresponding joint and then the function “set_dof_position_target(dof_ptr. pos)” will set the target position for this joint.

6. After running “Bittle_articulation.py”, you should be able to see Bittle resting at the ground plane as follows.

If nothing pumps up after running “Bittle_articulation.py”, you could also try the backup plan. You could take a look at the “load_stage.py” in the “simple” folder. You may need to combine “load_stage.py” and “Bittle_articulation.py”. The basic idea is using “load_stage.py” to load our “Bittle.usd” and using the same control strategy in “Bittle_articulation.py” to control all the joints.

4.2 Develop the Controller

Our next goal is to keep changing the “posvec” in 4.1.Step 5 so that Bittle will start to move. We could use the UDP communication structure and consider the controller as a sender while the simulator as a receiver. Since there are a lot of tutorials about how to use UDP in Python, I will focus on how to construct the “posvec” in our controller. You could directly use the pre-defined skills in instinct.h or take them as references.

Steps

1. In the very beginning, we need to define the sequence of the frames. The walking of Bittle is realized by moving through these frames periodically. You could get more information from Petoi’s official website https://bittle.petoi.com/11-tutorial-on-creating-new-skills;

2. Initialize and assign some parameters;

3. Send out the corresponding vectors. Noting that we need to change the sign of some joint values. Also, we could use a simple smoother function to avoid large torque changes for the driver. If you want to increase the value of “stepSmooth”, you may also need to change the value of “timeUpdate”, otherwise the time period for moving through all the frames will be too long. That may make Bittle unstable;

4. After you make all these changes, you should be able to see Bittle work in space after a small jump. You could modify the controller and move Bittle to the “Balance(stand)” gesture first then use the controller above. In this way, the jump will not happen. It is a good practice and I will leave it for you to finish. If you find your Bittle moves crazy, you may need to tune the drive gains in 3.Step 3. Also, you could play around these buttons to change the view and the background.