Quickstart

Create a scene

To create a scene you’ll use assets, robots and tools from the stacking demo package.

1. Download the stacking demo package from the Developer Portal .
2. Open the predefined scene main.usd from the package in NVIDIA Isaac Sim.

Add robot to scene

1. In the stage tree, navigate to MIR.
2. Right-click on Mounting.
3. Select Create → Wandelbots NOVA → Single Robot Model.
4. In the configuration window, select KUKA as manufacturer and KUKA_KR10_R900_2 as model.
5. Save the robot to a desired file location and confirm.
6. To reorganize the hierarchy, in the stage tree, move the imported robot to the workspace_kuka Xform to ensure a proper environment structure.

Add tool

To pick up objects, the robot requires a gripper:

Connect gripper to robot flange

1. In the stage tree, expand the Kuka_KR10_R900_2 hierarchy, navigate to Link_6 and expand it too.
2. To position the tool, drag the gripper schunk_coact_gripper from the content browser onto the tcp_flange Xform as seen in the video. The gripper aligns with the robot’s mounting flange.
3. To relocate the gripper into the correct layer of the robot assembly, drag the schunk_coact_gripper from the tcp_flange into the KUKA_KR10_R900_2 Xform.
4. To create the physical connection, select Link_6 first and then, with Ctrl key pressed, select the Gripper_Body prim of the tool, too.
5. Select Create → Physics → Joint → Fixed Joint.

Configure tool’s action graph

1. Right-click on the action graph of schunk_coact_gripper in the stage tree and select Open Graph.
2. In the action graph editor, search for the On IO Change node and drag it into the workspace.
3. Connect the Change output to Input Execution and the Boolean output to the Condition pin of the following logic node.
4. Select the On IO Change node and set the following properties in the Properties panel:\
   • Input Robot: Select the robot from the scene.
   • IO: Set to OUT#1.

Learn more on how to set up a parallel gripper or surface gripper as well as how to use OmniGraph nodes in the respective articles.

Add workpieces

Now that the gripper is ready to grab objects, let’s add some cubes to the scene that the robot can interact with.

1. From the asset folder drag and drop the workpiece.usd into your scene and into the workspace_kuka Xform. Once it is imported you can place it anywhere on the table.
2. Right click the workpiece prim and duplicate it twice.

Create ghost objects for pick and place poses

Ghost objects act as target poses for the robot’s TCP; tool center point. They help you visualize and plan movements before execution.

1. Right-click on the gripper in the stage tree.

2. Select Create → Wandelbots NOVA → Ghost Object.
   A poses prim is created under gripper.

3. Use the transform handles in the viewport to drag the ghost object to the precise location where the robot should pick up the cube.

4. Expand the poses prim and select the generated mesh.

5. Rename the mesh to PickPose_01.

6. Position PickPose_01 where the gripper should grab the first cube.
   A ghost overlay of the robot is displayed.
   If the overlay is visible, the position is reachable. If it disappears, the position is out of reach.

   If you can’t reach the position using the arrows in the viewport, use the Translation and Orientation fields in the Properties panel by entering precise coordinates.

7. Duplicate PickPose_01 and position the new PickPose_02 on the second cube.

8. Repeat step 7 for PickPose_03 on the third cube.

9. Duplicate one of the pick poses and rename the new pose to PlacePose_01.

10. Position the PlacePose_01 where the stack should start.

11. Duplicate PlacePose_01 to create PlacePose_02.

12. To stack the place poses, move PlacePose_02 vertically by the height of one cube.

13. Repeat step 11 and 12 for PlacePose_03, moving it up by another cube’s height to complete the stacking sequence.

Learn more about ghost objects and their use in ghost teaching.

Write robot program

To start stacking cubes, you need to write a robot program that defines the pick and place sequence using the ghost object poses as targets for the robot’s TCP.

Find the robot program in the examples folder  of the GitHub repository.

You can execute the program in two setups: either directly on the NOVA Cloud instance using the Visual Studio Code app from the App Store or via a local Visual Studio Code setup on your machine.

Start simulation

1. Create a virtual controller called kuka in your NOVA Cloud instance. Manufacturer: KUKA, model: kuka-kr10_r900_2.
2. In NVIDIA Isaac Sim, navigate to Wandelbots NOVA → Connected Instances and click Refresh.
   The virtual controller is displayed in the list.
   The virtual controller’s motion group is automatically matched with the Articulation in the scene.
3. To update the TCP on Wandelbots NOVA, in the stage tree, navigate to KUKA_KR10_R900_2 → schunk_coact_gripper → Gripper_Body → tcp_gripper.
4. Select Create → Wandelbots NOVA → TCP in NOVA.
   The TCP is added onto the virtual controller.
5. Start the NVIDIA Isaac Sim simulation with Run.
6. Start the robot program in Visual Studio Code and watch the simulation execute the stacking sequence in real-time.