# Quickstart

> Set up the Wandelbots Isaac Sim extension with Wandelbots NOVA and NVIDIA Isaac Sim. Start simulating your first robotics use case or explore a demo to test your setup.

Video: [/video/omniservice/quickstart/picking.webm](/video/omniservice/quickstart/picking.webm)

![Stacking boxes simulated in NVIDIA Isaac Sim](/assets/nova/omniservice/Build_a_Scene/Photo/Build-a-Scene.webp)

Video: [/video/omniservice/quickstart/placing2.webm](/video/omniservice/quickstart/placing2.webm)

Once you have installed the simulation environment and the Wandelbots NOVA extension for NVIDIA Isaac Sim,
you are ready to build your first simulation.
Explore the capabilities of Wandelbots NOVA and identify potential use cases for your projects with this quickstart guide.

### Prerequisites

You've completed the [installation of the Wandelbots NOVA extension for NVIDIA Isaac Sim](/simulation/installation) and your simulation environment
is connected to Wandelbots NOVA.

### Step 1: Create a scene

To create a scene you'll use assets, robots and tools from the stacking demo package.

1. Download the Stacking demo package from [Downloads](/nova-cloud/downloads) in NOVA Cloud.
2. Open the predefined scene `main.usd` from the package in NVIDIA Isaac Sim.

### Step 2: Add robot to scene

Video: [/video/omniservice/quickstart/download\_robot.webm](/video/omniservice/quickstart/download_robot.webm)

1. In the stage tree, navigate to `MIR`.
2. Right-click on `Mounting`.
3. Select `Create` → `Wandelbots NOVA` → `Single Robot Model`.
4. In the configuration window, select `KUKA` as manufacturer and `KUKA_KR10_R900_2` as model.
5. Save the robot to a desired file location and confirm.
6. To reorganize the hierarchy, in the stage tree, move the imported robot to the `workspace_kuka` Xform to ensure a proper environment structure.

### Step 3: Add tool

To pick up objects, the robot requires a gripper:

Video: [/video/omniservice/quickstart/import\_gripper.webm](/video/omniservice/quickstart/import_gripper.webm)

#### Connect gripper to robot flange

1. In the stage tree, expand the `Kuka_KR10_R900_2` hierarchy, navigate to `Link_6` and expand it too.
2. To position the tool, drag the gripper `schunk_coact_gripper` from the content browser onto the `tcp_flange` Xform as seen in the video. The gripper aligns with the robot's mounting flange.
3. To relocate the gripper into the correct layer of the robot assembly, drag the `schunk_coact_gripper` from the `tcp_flange` into the `KUKA_KR10_R900_2` Xform.
4. To create the physical connection, select `Link_6` first and then, with `Ctrl` key pressed, select the `Gripper_Body` prim of the tool, too.
5. Select `Create` → `Physics` → `Joint` → `Fixed Joint`.

#### Configure tool's action graph

1. Right-click on the action graph of `schunk_coact_gripper` in the stage tree and select `Open Graph`.
2. In the action graph editor, search for the `On IO Change` node and drag it into the workspace.
3. Connect the `Change` output to `Input Execution` and the `Boolean` output to the `Condition` pin of the following logic node.
4. Select the `On IO Change` node and set the following properties in the `Properties` panel:\\
   - `Input Robot`: Select the robot from the scene.
   - `IO`: Set to `OUT#1`.

Learn more on how to set up a [parallel gripper](/simulation/build-scene/parallel-gripper) or [surface gripper](/simulation/build-scene/surface-gripper) as well as [how to use OmniGraph nodes](/simulation/omnigraph-nodes/connect-ios) in the respective articles.

### Step 4: Add workpieces

Now that the gripper is ready to grab objects, let's add some cubes to the scene that the robot can interact with.

Video: [/video/omniservice/quickstart/import\_workpiece.webm](/video/omniservice/quickstart/import_workpiece.webm)

1. From the asset folder drag and drop the `workpiece.usd` into your scene and into the `workspace_kuka` _Xform_. Once it is imported you can place it anywhere on the table.
2. Right click the workpiece prim and duplicate it twice.

### Step 5: Create ghost objects for pick and place poses

Video: [/video/omniservice/quickstart/create\_ghost\_objects.webm](/video/omniservice/quickstart/create_ghost_objects.webm)

Ghost objects act as target poses for the robot's [TCP; tool center point](/nova/latest/nova-api/core-concepts/robotics-basics#tool-center-point-tcp-and-payload).
They help you visualize and plan movements before execution.

1. Right-click on the gripper in the stage tree.

2. Select `Create` → `Wandelbots NOVA` → `Ghost Object`.\
   → A `poses` prim is created under gripper.

3. Use the transform handles in the viewport to drag the ghost object to the precise location where the robot should pick up the cube.

4. Expand the `poses` prim and select the generated mesh.

5. Rename the mesh to `PickPose_01`.

6. Position `PickPose_01` where the gripper should grab the first cube.\
   → A ghost overlay of the robot is displayed.\
   If the overlay is visible, the position is reachable.
   If it disappears, the position is out of reach.

   > **Info:** If you can't reach the position using the arrows in the viewport, use the `Translation` and `Orientation` fields in the `Properties` panel by entering precise coordinates.

7. Duplicate `PickPose_01` and position the new `PickPose_02` on the second cube.

8. Repeat step 7 for `PickPose_03` on the third cube.

9. Duplicate one of the pick poses and rename the new pose to `PlacePose_01`.

10. Position the `PlacePose_01` where the stack should start.

11. Duplicate `PlacePose_01` to create `PlacePose_02`.

12. To stack the place poses, move `PlacePose_02` vertically by the height of one cube.

13. Repeat step 11 and 12 for `PlacePose_03`, moving it up by another cube's height to complete the stacking sequence.

Learn more about ghost objects and their use in [ghost teaching](/simulation/ghost-teaching).

### Step 6: Write robot program

To start stacking cubes, you need to write a robot program that defines the pick and place sequence using the ghost object poses as targets
for the robot's TCP.

Use the robot program `stacking.py` (Stacking demo package > scripts folder) from [Downloads](/nova-cloud/downloads) in NOVA Cloud:

> **Info:** The name of the gripper's TCP in NVIDIA Isaac Sim must be identical to the TCP name defined in your robot program (line 95). Copy the TCP name from your code and rename the gripper's TCP object in the stage tree accordingly (or vice versa) to avoid connection errors.
> The robot's name in the program should remain unchanged.

**Show robot program** ([source](https://github.com/wandelbotsgmbh/wandelbots-isaacsim-extension/blob/main/examples/nova_sdk/quickstart_stacking_demo.py))

```python
# package imports
import nova
import wandelbots_isaacsim_api as isaac_sim_api

import wandelbots_isaacsim_api.trajectory as trajectory_utils
from nova import run_program
from nova import api
from nova.actions import cartesian_ptp, io_write, linear, wait
from nova.actions.io import WriteAction
from nova.actions.mock import WaitAction
from nova.actions.motions import CartesianPTP, Linear
from nova.cell import virtual_controller
from nova.cell.cell import Cell
from nova.cell.controller import Controller
from nova.cell.motion_group import MotionGroup
from nova.types.motion_settings import MotionSettings
from nova.types.pose import Pose
from wandelbots_isaacsim_api.api.teaching_api import TeachingApi
from wandelbots_isaacsim_api.models.ghost_object import GhostObject
import logging

logger = logging.getLogger(__name__)

# parameter definitions
isaacsim_ip_address = "<add your_isaac_sim_ip_address_here>"
omniservice_host = f"http://{isaacsim_ip_address}:8011/omniservice/api/v2"
controller_name = "kuka"
robot_prim_path = "/World/cell/workspace_kuka/KUKA_KR10_R900_2"
manufacturer = api.models.Manufacturer.KUKA
robot_model = "kuka-kr10_r900_2"
gripper_signal = "OUT#1"
motion_group_num = 0

# helper function to get poses from ghost objects in isaac sim
async def get_poses_from_ghost_objects(
    isaac_sim_api_url: str, robot_prim_path: str
) -> dict[str, list[Pose]]:
    """
    Helper function to get pick and place poses from ghost objects in the simulation model.
    Returns a dictionary mapping each unique ghost object name to a list of Pose objects.
    """
    async with isaac_sim_api.ApiClient(
        configuration=isaac_sim_api.Configuration(host=isaac_sim_api_url)
    ) as isaac_sim_api_client:
        teaching_api: TeachingApi = isaac_sim_api.TeachingApi(
            api_client=isaac_sim_api_client
        )
        ghost_objects: list[GhostObject] | None = None
        ghost_objects = await teaching_api.list_ghost_objects(
            relative_to_prim=robot_prim_path
        )

        # Create a dictionary mapping each unique ghost object name to a list of Pose objects
        poses_dict: dict[str, list[Pose]] = {}
        for ghost_obj in ghost_objects:
            name = ghost_obj.name
            pose_obj = Pose(tuple(ghost_obj.pose.pose))
            if name not in poses_dict:
                poses_dict[name] = []
            poses_dict[name].append(pose_obj)
        return poses_dict


# configure the robot program
@nova.program(
    id="stacking_demo",
    name="Stacking Demo",
    viewer=trajectory_utils.TrajectoryViewer(
        omniverse_host=omniservice_host,
        motion_group_prim_paths={
            f"{motion_group_num}@{controller_name}": robot_prim_path,
        },
    ),
    preconditions=nova.ProgramPreconditions(
        controllers=[
            virtual_controller(
                name=controller_name,
                manufacturer=manufacturer,
                type=robot_model,
            )
        ],
        cleanup_controllers=False,
    )
)
async def start(ctx: nova.ProgramContext) -> None:
    """Main robot control function."""

    # get cell, controller, and motion group instances from the Nova instance
    cell: Cell = ctx.cell
    controller: Controller = await cell.controller(controller_name)
    motion_group: MotionGroup = controller[motion_group_num]

    # define motion speed and tcp
    fast: MotionSettings = MotionSettings(tcp_velocity_limit=250)
    tcp = "schunk_coact_gripper"

    # get ghost objects from isaac sim
    robot_poses: dict[str, list[Pose]] = await get_poses_from_ghost_objects(
        isaac_sim_api_url=omniservice_host,
        robot_prim_path=robot_prim_path,
    )

    # iterate over ghost objects and run pick and place motion commands
    pick_poses = [pose for key, poses in robot_poses.items() if "PickPose" in key for pose in poses]
    place_poses = [pose for key, poses in robot_poses.items() if "PlacePose" in key for pose in poses]
    for (pick_pose, place_pose) in zip(pick_poses, place_poses):
        # define robot motion sequence
        actions: list[CartesianPTP | Linear | WriteAction | WaitAction] = [
            cartesian_ptp(
                target=pick_pose @ Pose(0, 0, -200, 0, 0, 0),
                settings=fast,
            ),
            linear(target=pick_pose, settings=fast),
            io_write(key=gripper_signal, value=True),
            wait(wait_for_in_seconds=2),
            linear(
                target=pick_pose @ Pose(0, 0, -200, 0, 0, 0),
                settings=fast,
            ),
            cartesian_ptp(
                target=place_pose @ Pose(0, 0, -200, 0, 0, 0),
                settings=fast,
            ),
            linear(target=place_pose, settings=fast),
            io_write(key=gripper_signal, value=False),
            wait(wait_for_in_seconds=2),
            linear(
                target=place_pose @ Pose(0, 0, -200, 0, 0, 0),
                settings=fast,
            ),

        ]
        await motion_group.plan_and_execute(actions, tcp)

# execute program
if __name__ == "__main__":
    run_program(program=start)
```

You can execute the program in two setups: either directly on the [NOVA cloud instance](/nova-cloud/cloud-instances) using the Visual Studio Code app from the App Store or
via a local Visual Studio Code setup on your machine.

**Local Setup**

1. Open Visual Studio Code and create a new folder for your project.
2. Install `uv`.
3. Install the [NOVA Python SDK](https://github.com/wandelbotsgmbh/wandelbots-nova) and the Wandelbots Isaac Sim Extension API client:
   ```bash
   uv init
   uv add wandelbots-nova wandelbots-isaacsim-api
   ```
4. Create a new file named `main.py` within your project folder and paste the example code or your own code.
5. In `main.py`, go to line 25 and enter the IP address of the host machine where Isaac Sim is currently running on.
6. Create a new file named `.env` in `my_project` and add the following line to the file:
   ```python
   NOVA_API = "http://<Your_NOVA_Instance_URL>" # replace `Your_NOVA_Instance_URL` with the actual URL of your running NOVA cloud instance.
   NOVA_ACCESS_TOKEN = "My_NOVA_Access_Token" # replace `My_NOVA_Access_Token` with the actual access token of your running NOVA cloud instance.
   ```

The instance URL and access token come from NOVA Cloud, see [Cloud instances](/nova-cloud/cloud-instances).
Learn more about how to set up a local Visual Studio Code environment and connect it to a NOVA cloud instance [here](https://github.com/wandelbotsgmbh/wandelbots-nova?tab=readme-ov-file#installation).

**Cloud Setup**

1. Open the Wandelbots NOVA interface and install Visual Studio Code via the App Store.
2. Open Visual Studio Code from the home screen.
3. Open the folder or project named `your_nova_app`.
4. Open the `start_here.py` file and paste the example code or your own code.
5. Open the Visual Studio Code terminal and run the following commands to add the necessary libraries using `uv`:
   ```bash
   uv add wandelbots-isaacsim-api
   ```

### Step 7: Start simulation

1. Create a virtual controller called `kuka` in your NOVA Cloud instance. Manufacturer: KUKA, model: `kuka-kr10_r900_2`.
2. In NVIDIA Isaac Sim, navigate to `Wandelbots NOVA` → `Connected Instances` and click `Refresh`.\
   → The virtual controller is displayed in the list.
   → The virtual controller's motion group is automatically matched with the `Articulation` in the scene.
3. To update the TCP on Wandelbots NOVA, in the stage tree, navigate to `KUKA_KR10_R900_2` → `schunk_coact_gripper` → `Gripper_Body` → `tcp_gripper`.
4. Select `Create` → `Wandelbots NOVA` → `TCP in NOVA`.\
   → The TCP is added onto the virtual controller.
5. Start the NVIDIA Isaac Sim simulation with `Run`.
6. Start the robot program in Visual Studio Code and watch the simulation execute the stacking sequence in real-time.

> **Info:** If you want to use your own tool, adjust the TCP values accordingly. Pass the original tool file to a CAD tool and calculate the TCP values.\
> If the TCP has angles, additional calculation is required which the Wandelbots Customer Success is happy to help you with.

Video: [/video/omniservice/quickstart/start\_simulation.webm](/video/omniservice/quickstart/start_simulation.webm)

Congrats! You've successfully set up the stacking demo and simulated the stacking scenario.\
To step it up a notch, try modifying the tool or the TCP position. Let us know how it went!