IK code

[tabula-rosa]

Adds the "IK" capability group to IsaacGym. Closes #27.

[jacky-liang]

Thanks for working on this :) Couple points:

• Could you fold in the IK functionality as part of the Franka class and implement it as set_ee_transform and set_ee_transform_delta? You can rename the current methods to set_ee_transform_target and set_ee_transform_delta_target. This way the user does not need to know how the IK is implemented. We should hide the additional data structures used by rtk away from the user.
• These functions should also take care of the custom_ee_offset and custom_ee_rb_name attributes in GymFranka - it's common for ppl to use custom end-effectors in the URDF w/ different offsets. For example wrist mounted cameras would induce an ee offset.
• Make imports of rtk in GymFranka optional by using try/catch and logging a warning if it's not installed.
• How big is rtk? if it's not that big i'm actually fine w/ making this a required dependency.

[tabula-rosa]

@jacky-liang Could you take a look at this? Some things:

• Example is run by: python examples/franka_pick_block_ik.py
• Could you give a use case or example file for when a custom ee is used and how GymFranka usually takes care of that?
• For set_ee_transform_delta, were you thinking of delta in the EE local frame or world frame? (set_ee_transform is expecting EE transforms in world frame). Either way unless this is strictly necessary for this PR, I would suggest adding it as a separate PR later
• The total size of packages is about 5M (roboticstoolbox: 3.4M, spatialmath: 1.5M) so it's pretty small. Compare this to numpy which is 28M. Isaac Gym itself is 380M (at least, according to what pip is checking). So, I can keep the optional ik extras, or we can just add it as a core dependency. With a good joint init (which I've added by default), it seems robust enough to make a core dependency so users will have IK functionality as a part of the core library.

[jacky-liang]

yeah i'm ok w/ including rtb in the core dependencies.

with the franka urdf we're using, the "end-effector" frame is near the wrist, which is a little inconvenient to use. To move the "end-effector" frame to the middle of the gripper, we use the hardcoded self._gripper_offset during getting ee transforms and setting ee attractor targets.

As you can see in those lines, an additional self._custom_ee_offset is also used in case the user wants to move the ee frame somewhere else, relative to the middle of the gripper. this offset is parsed from the config as a 3 vector for translation here.

I'm not sure what the end-effector frame is for rtb's panda urdf. Ideally, instead of using their urdf, we have them parse the same urdf that IG uses. Assuming the end-effector frame for the rtb panda is also near the wrist, to handle the custom ee offset, you just need to right-multiply the desired ee transform like: ee_transform = ee_transform * self._ee_tool_offset.inverse() * self._gripper_offset.inverse() inside joints_from_ee_transform_ik. If the end-effector frame for the rtb panda is not the same as the one in the urdf we use, then you'd need to find their relative transform and multiply accordingly.

for set_ee_transform_delta, you can pretty much just copy the code from here, but replace the call at the end to use set_ee_transform instead of set_ee_transform_target

also rename joints_from_ee_transform_ik to get_joints_from_ee_transform_ik

[tabula-rosa]

Okay, thanks, that helps me better understand re: custom EEs.

It appears the RTB model is hardcoding a particular EE offset (https://github.com/petercorke/robotics-toolbox-python/blob/51aa8bbb3663a7c815f9880d538d61e7c85bc470/roboticstoolbox/models/URDF/Panda.py#L48). My preference is to try and have RTB load the URDF specified by the IG user to avoid a situation where there could be a URDF mismatch. After briefly looking into this, the URDF loading tools by RTB seem to assume a certain folder structure, but there could be a workaround and I will look into it later.

[jacky-liang]

Okay, thanks, that helps me better understand re: custom EEs.

It appears the RTB model is hardcoding a particular EE offset (https://github.com/petercorke/robotics-toolbox-python/blob/51aa8bbb3663a7c815f9880d538d61e7c85bc470/roboticstoolbox/models/URDF/Panda.py#L48). My preference is to try and have RTB load the URDF specified by the IG user to avoid a situation where there could be a URDF mismatch. After briefly looking into this, the URDF loading tools by RTB seem to assume a certain folder structure, but there could be a workaround and I will look into it later.

Lol we use the same offset

if loading custom urdf is too much of a hassle you can just assume the offsets are the same in which case you can ignore self._gripper_offset.inverse() in the ee_transform calculation