Skip to content

Commit 902bded

Browse files
Milad-Rakhsha-NVMilad-Rakhsha-NV
authored
[Newton] Adds policy transfer script for sim2sim transfer from Newton to physX (#3565)
# Description This PR adds a play script to physX-based IsaacLab to make it possible to play a Newton-based trained policy. Tested environments are H1/G1/Anymal-D but other exisiting Newton environment should transfer as well. <!-- Thank you for your interest in sending a pull request. Please make sure to check the contribution guidelines. Link: https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html 💡 Please try to keep PRs small and focused. Large PRs are harder to review and merge. --> Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. Fixes # (issue) <!-- As a practice, it is recommended to open an issue to have discussions on the proposed pull request. This makes it easier for the community to keep track of what is being developed or added, and if a given feature is demanded by more than one party. --> ## Type of change <!-- As you go through the list, delete the ones that are not applicable. --> - Bug fix (non-breaking change which fixes an issue) - New feature (non-breaking change which adds functionality) - Breaking change (existing functionality will not work without user modification) - Documentation update ## Screenshots Please attach before and after screenshots of the change if applicable. <!-- Example: | Before | After | | ------ | ----- | | _gif/png before_ | _gif/png after_ | To upload images to a PR -- simply drag and drop an image while in edit mode and it should upload the image directly. You can then paste that source into the above before/after sections. --> ## Checklist - [ ] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [ ] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [ ] I have added my name to the `CONTRIBUTORS.md` or my name already exists there <!-- As you go through the checklist above, you can mark something as done by putting an x character in it For example, - [x] I have done this task - [ ] I have not done this task --> --------- Signed-off-by: Milad Rakhsha <[email protected]>
1 parent faa96df commit 902bded

File tree

6 files changed

+576
-23
lines changed

6 files changed

+576
-23
lines changed

‎docs/source/experimental-features/newton-physics-integration/sim-to-sim.rst‎

Lines changed: 91 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -2,53 +2,55 @@
22

33
Sim-to-Sim Policy Transfer
44
==========================
5-
This section provides examples of sim-to-sim policy transfer using the Newton backend. Sim-to-sim transfer is an essential step before real robot deployment because it verifies that policies work across different simulators. Policies that pass sim-to-sim verification are much more likely to succeed on real robots.
5+
This section provides examples of sim-to-sim policy transfer between PhysX and Newton backends. Sim-to-sim transfer is an essential step before real robot deployment because it verifies that policies work across different simulators. Policies that pass sim-to-sim verification are much more likely to succeed on real robots.
66

77

88
Overview
99
--------
1010

11-
This guide shows how to run a PhysX-trained policy on the Newton backend. While the method works for any robot and physics engine, it has only been tested with Unitree G1, Unitree H1, and ANYmal-D robots using PhysX-trained policies.
11+
This guide shows how to transfer policies between PhysX and Newton backends in both directions. The main challenge is that different physics engines may parse the same robot model with different joint and link ordering.
1212

13-
PhysX-trained policies expect joints and links in a specific order determined by how PhysX parses the robot model. However, Newton may parse the same robot with different joint and link ordering.
13+
Policies trained in one backend expect joints and links in a specific order determined by how that backend parses the robot model. When transferring to another backend, the joint ordering may be different, requiring remapping of observations and actions.
1414

1515
In the future, we plan to solve this using **robot schema** that standardizes joint and link ordering across different backends.
1616

17-
Currently, we solve this by remapping observations and actions using joint mappings defined in YAML files. These files specify joint names in both PhysX order (source) and Newton order (target). During policy execution, we use this mapping to reorder observations and actions so they work correctly with Newton.
17+
Currently, we solve this by remapping observations and actions using joint mappings defined in YAML files. These files specify joint names in both source and target backend orders. During policy execution, we use this mapping to reorder observations and actions so they work correctly with the target backend.
18+
19+
The method has been tested with Unitree G1, Unitree Go2, Unitree H1, and ANYmal-D robots for both transfer directions.
1820

1921

2022
What you need
2123
~~~~~~~~~~~~~
2224

23-
- A policy checkpoint trained with PhysX (RSL-RL).
24-
- A joint mapping YAML for your robot under ``scripts/newton_sim2sim/mappings/``.
25-
- The provided player script: ``scripts/newton_sim2sim/rsl_rl_transfer.py``.
25+
- A policy checkpoint trained with either PhysX or Newton (RSL-RL).
26+
- A joint mapping YAML for your robot under ``scripts/sim2sim_transfer/config/``.
27+
- The provided player script: ``scripts/sim2sim_transfer/rsl_rl_transfer.py``.
2628

2729
To add a new robot, create a YAML file with two lists where each joint name appears exactly once in both:
2830

2931
.. code-block:: yaml
3032
3133
# Example structure
32-
source_joint_names: # PhysX joint order
34+
source_joint_names: # Source backend joint order
3335
- joint_1
3436
- joint_2
3537
# ...
36-
target_joint_names: # Newton joint order
38+
target_joint_names: # Target backend joint order
3739
- joint_1
3840
- joint_2
3941
# ...
4042
4143
The script automatically computes the necessary mappings for locomotion tasks.
4244

4345

44-
How to run
45-
~~~~~~~~~~
46+
PhysX-to-Newton Transfer
47+
~~~~~~~~~~~~~~~~~~~~~~~~
4648

47-
Use this command template to run a PhysX-trained policy with Newton:
49+
To run a PhysX-trained policy with the Newton backend, use this command template:
4850

4951
.. code-block:: bash
5052
51-
./isaaclab.sh -p scripts/newton_sim2sim/rsl_rl_transfer.py \
53+
./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
5254
--task=<TASK_ID> \
5355
--num_envs=32 \
5456
--checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
@@ -60,40 +62,106 @@ Here are examples for different robots:
6062

6163
.. code-block:: bash
6264
63-
./isaaclab.sh -p scripts/newton_sim2sim/rsl_rl_transfer.py \
65+
./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
6466
--task=Isaac-Velocity-Flat-G1-v0 \
6567
--num_envs=32 \
6668
--checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
67-
--policy_transfer_file scripts/newton_sim2sim/mappings/sim2sim_g1.yaml
69+
--policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_g1.yaml
6870
6971
7072
2. Unitree H1
7173

7274

7375
.. code-block:: bash
7476
75-
./isaaclab.sh -p scripts/newton_sim2sim/rsl_rl_transfer.py \
77+
./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
7678
--task=Isaac-Velocity-Flat-H1-v0 \
7779
--num_envs=32 \
7880
--checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
79-
--policy_transfer_file scripts/newton_sim2sim/mappings/sim2sim_h1.yaml
81+
--policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_h1.yaml
82+
83+
84+
3. Unitree Go2
8085

86+
.. code-block:: bash
8187
82-
3. ANYmal-D
88+
./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
89+
--task=Isaac-Velocity-Flat-Go2-v0 \
90+
--num_envs=32 \
91+
--checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
92+
--policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_go2.yaml
93+
94+
95+
4. ANYmal-D
8396

8497

8598
.. code-block:: bash
8699
87-
./isaaclab.sh -p scripts/newton_sim2sim/rsl_rl_transfer.py \
100+
./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
88101
--task=Isaac-Velocity-Flat-Anymal-D-v0 \
89102
--num_envs=32 \
90103
--checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
91-
--policy_transfer_file scripts/newton_sim2sim/mappings/sim2sim_anymal_d.yaml
104+
--policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_anymal_d.yaml
105+
106+
Note that to run this, you need to checkout the Newton-based branch of IsaacLab such as ``feature/newton``.
107+
108+
Newton-to-PhysX Transfer
109+
~~~~~~~~~~~~~~~~~~~~~~~~
110+
111+
To transfer Newton-trained policies to PhysX-based IsaacLab, use the reverse mapping files:
112+
113+
Here are examples for different robots:
114+
115+
1. Unitree G1
116+
117+
.. code-block:: bash
118+
119+
./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
120+
--task=Isaac-Velocity-Flat-G1-v0 \
121+
--num_envs=32 \
122+
--checkpoint <PATH_TO_NEWTON_CHECKPOINT> \
123+
--policy_transfer_file scripts/sim2sim_transfer/config/newton_to_physx_g1.yaml
124+
125+
126+
2. Unitree H1
127+
128+
.. code-block:: bash
129+
130+
./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
131+
--task=Isaac-Velocity-Flat-H1-v0 \
132+
--num_envs=32 \
133+
--checkpoint <PATH_TO_NEWTON_CHECKPOINT> \
134+
--policy_transfer_file scripts/sim2sim_transfer/config/newton_to_physx_h1.yaml
135+
136+
137+
3. Unitree Go2
138+
139+
.. code-block:: bash
140+
141+
./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
142+
--task=Isaac-Velocity-Flat-Go2-v0 \
143+
--num_envs=32 \
144+
--checkpoint <PATH_TO_NEWTON_CHECKPOINT> \
145+
--policy_transfer_file scripts/sim2sim_transfer/config/newton_to_physx_go2.yaml
146+
147+
148+
4. ANYmal-D
149+
150+
.. code-block:: bash
151+
152+
./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
153+
--task=Isaac-Velocity-Flat-Anymal-D-v0 \
154+
--num_envs=32 \
155+
--checkpoint <PATH_TO_NEWTON_CHECKPOINT> \
156+
--policy_transfer_file scripts/sim2sim_transfer/config/newton_to_physx_anymal_d.yaml
92157
158+
The key difference is using the ``newton_to_physx_*.yaml`` mapping files instead of ``physx_to_newton_*.yaml`` files. Also note that you need to checkout a PhysX-based IsaacLab branch such as ``main``.
93159

94-
Notes and limitations
160+
Notes and Limitations
95161
~~~~~~~~~~~~~~~~~~~~~
96162

97-
- This transfer method has only been tested with Unitree G1, Unitree H1, and ANYmal-D using PhysX-trained policies.
98-
- The observation remapping assumes a locomotion layout with base observations followed by joint observations. For different observation layouts, you'll need to modify ``scripts/newton_sim2sim/policy_mapping.py``.
163+
- Both transfer directions have been tested with Unitree G1, Unitree Go2, Unitree H1, and ANYmal-D robots.
164+
- PhysX-to-Newton transfer uses ``physx_to_newton_*.yaml`` mapping files.
165+
- Newton-to-PhysX transfer requires the corresponding ``newton_to_physx_*.yaml`` mapping files and the PhysX branch of IsaacLab.
166+
- The observation remapping assumes a locomotion layout with base observations followed by joint observations. For different observation layouts, you'll need to modify the ``get_joint_mappings`` function in ``scripts/sim2sim_transfer/rsl_rl_transfer.py``.
99167
- When adding new robots or backends, make sure both source and target have identical joint names, and that the YAML lists reflect how each backend orders these joints.

‎scripts/sim2sim_transfer/config/newton_to_physx_anymal_d.yaml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
2+
# All rights reserved.
3+
#
4+
# SPDX-License-Identifier: BSD-3-Clause
5+
6+
# Joint names in the source physics engine where policy is trained (Newton)
7+
source_joint_names:
8+
- "LF_HAA"
9+
- "LF_HFE"
10+
- "LF_KFE"
11+
- "LH_HAA"
12+
- "LH_HFE"
13+
- "LH_KFE"
14+
- "RF_HAA"
15+
- "RF_HFE"
16+
- "RF_KFE"
17+
- "RH_HAA"
18+
- "RH_HFE"
19+
- "RH_KFE"
20+
21+
# Joint names in the target physics engine where policy is deployed (PhysX)
22+
target_joint_names:
23+
- "LF_HAA"
24+
- "LH_HAA"
25+
- "RF_HAA"
26+
- "RH_HAA"
27+
- "LF_HFE"
28+
- "LH_HFE"
29+
- "RF_HFE"
30+
- "RH_HFE"
31+
- "LF_KFE"
32+
- "LH_KFE"
33+
- "RF_KFE"
34+
- "RH_KFE"

‎scripts/sim2sim_transfer/config/newton_to_physx_g1.yaml‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
2+
# All rights reserved.
3+
#
4+
# SPDX-License-Identifier: BSD-3-Clause
5+
6+
# Joint names in the source physics engine where policy is trained (Newton)
7+
source_joint_names:
8+
- "left_hip_pitch_joint"
9+
- "left_hip_roll_joint"
10+
- "left_hip_yaw_joint"
11+
- "left_knee_joint"
12+
- "left_ankle_pitch_joint"
13+
- "left_ankle_roll_joint"
14+
- "right_hip_pitch_joint"
15+
- "right_hip_roll_joint"
16+
- "right_hip_yaw_joint"
17+
- "right_knee_joint"
18+
- "right_ankle_pitch_joint"
19+
- "right_ankle_roll_joint"
20+
- "torso_joint"
21+
- "left_shoulder_pitch_joint"
22+
- "left_shoulder_roll_joint"
23+
- "left_shoulder_yaw_joint"
24+
- "left_elbow_pitch_joint"
25+
- "left_elbow_roll_joint"
26+
- "left_five_joint"
27+
- "left_six_joint"
28+
- "left_three_joint"
29+
- "left_four_joint"
30+
- "left_zero_joint"
31+
- "left_one_joint"
32+
- "left_two_joint"
33+
- "right_shoulder_pitch_joint"
34+
- "right_shoulder_roll_joint"
35+
- "right_shoulder_yaw_joint"
36+
- "right_elbow_pitch_joint"
37+
- "right_elbow_roll_joint"
38+
- "right_five_joint"
39+
- "right_six_joint"
40+
- "right_three_joint"
41+
- "right_four_joint"
42+
- "right_zero_joint"
43+
- "right_one_joint"
44+
- "right_two_joint"
45+
46+
# Joint names in the target physics engine where policy is deployed (PhysX)
47+
target_joint_names:
48+
- "left_hip_pitch_joint"
49+
- "right_hip_pitch_joint"
50+
- "torso_joint"
51+
- "left_hip_roll_joint"
52+
- "right_hip_roll_joint"
53+
- "left_shoulder_pitch_joint"
54+
- "right_shoulder_pitch_joint"
55+
- "left_hip_yaw_joint"
56+
- "right_hip_yaw_joint"
57+
- "left_shoulder_roll_joint"
58+
- "right_shoulder_roll_joint"
59+
- "left_knee_joint"
60+
- "right_knee_joint"
61+
- "left_shoulder_yaw_joint"
62+
- "right_shoulder_yaw_joint"
63+
- "left_ankle_pitch_joint"
64+
- "right_ankle_pitch_joint"
65+
- "left_elbow_pitch_joint"
66+
- "right_elbow_pitch_joint"
67+
- "left_ankle_roll_joint"
68+
- "right_ankle_roll_joint"
69+
- "left_elbow_roll_joint"
70+
- "right_elbow_roll_joint"
71+
- "left_five_joint"
72+
- "left_three_joint"
73+
- "left_zero_joint"
74+
- "right_five_joint"
75+
- "right_three_joint"
76+
- "right_zero_joint"
77+
- "left_six_joint"
78+
- "left_four_joint"
79+
- "left_one_joint"
80+
- "right_six_joint"
81+
- "right_four_joint"
82+
- "right_one_joint"
83+
- "left_two_joint"
84+
- "right_two_joint"

‎scripts/sim2sim_transfer/config/newton_to_physx_go2.yaml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
2+
# All rights reserved.
3+
#
4+
# SPDX-License-Identifier: BSD-3-Clause
5+
6+
# Joint names in the source physics engine where policy is trained (Newton)
7+
source_joint_names:
8+
- "FL_hip_joint"
9+
- "FL_thigh_joint"
10+
- "FL_calf_joint"
11+
- "FR_hip_joint"
12+
- "FR_thigh_joint"
13+
- "FR_calf_joint"
14+
- "RL_hip_joint"
15+
- "RL_thigh_joint"
16+
- "RL_calf_joint"
17+
- "RR_hip_joint"
18+
- "RR_thigh_joint"
19+
- "RR_calf_joint"
20+
# Joint names in the target physics engine where policy is deployed (PhysX)
21+
target_joint_names:
22+
- "FL_hip_joint"
23+
- "FR_hip_joint"
24+
- "RL_hip_joint"
25+
- "RR_hip_joint"
26+
- "FL_thigh_joint"
27+
- "FR_thigh_joint"
28+
- "RL_thigh_joint"
29+
- "RR_thigh_joint"
30+
- "FL_calf_joint"
31+
- "FR_calf_joint"
32+
- "RL_calf_joint"
33+
- "RR_calf_joint"

‎scripts/sim2sim_transfer/config/newton_to_physx_h1.yaml‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
2+
# All rights reserved.
3+
#
4+
# SPDX-License-Identifier: BSD-3-Clause
5+
6+
# Joint names in the source physics engine where policy is trained (Newton)
7+
source_joint_names:
8+
- "left_hip_yaw"
9+
- "left_hip_roll"
10+
- "left_hip_pitch"
11+
- "left_knee"
12+
- "left_ankle"
13+
- "right_hip_yaw"
14+
- "right_hip_roll"
15+
- "right_hip_pitch"
16+
- "right_knee"
17+
- "right_ankle"
18+
- "torso"
19+
- "left_shoulder_pitch"
20+
- "left_shoulder_roll"
21+
- "left_shoulder_yaw"
22+
- "left_elbow"
23+
- "right_shoulder_pitch"
24+
- "right_shoulder_roll"
25+
- "right_shoulder_yaw"
26+
- "right_elbow"
27+
28+
# Joint names in the target physics engine where policy is deployed (PhysX)
29+
target_joint_names:
30+
- "left_hip_yaw"
31+
- "right_hip_yaw"
32+
- "torso"
33+
- "left_hip_roll"
34+
- "right_hip_roll"
35+
- "left_shoulder_pitch"
36+
- "right_shoulder_pitch"
37+
- "left_hip_pitch"
38+
- "right_hip_pitch"
39+
- "left_shoulder_roll"
40+
- "right_shoulder_roll"
41+
- "left_knee"
42+
- "right_knee"
43+
- "left_shoulder_yaw"
44+
- "right_shoulder_yaw"
45+
- "left_ankle"
46+
- "right_ankle"
47+
- "left_elbow"
48+
- "right_elbow"

0 commit comments

Comments
 (0)