Start simulating with Omniservice API

After you have successfully set up a cell with NOVA API or in settings, you are now ready to access the Omniservice API. How you access the Omniservice API depends on the location of your NVIDIA Isaac Sim™ installation:

Local installation

If you installed the Isaac Sim™ extension (Omniservice) locally, access the API via http://localhost:8211/omniservice/api.

Remote installation

If you installed NVIDIA Isaac Sim™ on a remote machine, the Omniservice API needs to be exposed by a remotly accessible URL. We recommend Microsoft DevTunnel (opens in a new tab) for this purpose. The following steps are required to configure a DevTunnel:

winget install Microsoft.DevTunnel

devtunnel login

devtunnel host -a --port 8211

Authenticate with Wandelbots NOVA

If you want to connect NVIDIA Isaac Sim™ to a Wandelbots NOVA cloud instance, you need to authenticate with credentials provided when creating a cloud instance.

To authenticate with Wandelbots NOVA, use the POST /omniservice/authenticate endpoint. The request body should contain the following information:

{
  "host": string,         // The hostname of your NOVA instance without the protocol, e.g. `<instance-id>.instance.wandelbots.io`
  "is_secured: boolean,   // Set to true if your NOVA instance uses HTTPS
  "access_token": string,     // The access token of your NOVA instance
}

The information is cached by the Isaac Sim™ extension (Omniservice), thus you only need to authenticate once per session. When you restart Isaac Sim™ or the extension and thus terminate the session, you need to authenticate again.

Connect a robot

Currently, the following robot brands are supported by Wandelbots NOVA and can be simulated in an Isaac Sim™ scene: KUKA, ABB, FANUC, UniversalRobots and Yaskawa.

Set up a connection to one of the supported robots by completing the following steps. In this example, we use a UR5e with a parallel gripper attached to the robot flange as fixed joint.

{
  identifier: string,       // Name your robot
  host: string,             // The IP address of your NOVA instance
  cell: string,             // The name of your NOVA cell
  controller_id: string,    // The controller name of your physical robot, e.g. ur5e
  motion_group_id: integer, // The motion group of your robot instance, usually 0
  prim_path: string,        // The prim path of the robot (articulation) in the Isaac Sim™ ccene
}

{
  identifier: string,    // Choose a name, e.g. `ur5e_state_stream`
  type: string,          // The type of the stream, in this case RobotStateConnector
  robot: string,         // The identifier of your created robot, e.g. `ur5e`
  robot_state: string    // The update rate in ms
}

Periphery

With the Isaac Sim™ extension (Omniservice) you can also connect to tools controlled by the robot controller or by API calls via I/Os.

Currently, three types of periphery are supported:

• Surface grippers
• Parallel grippers
• Conveyor belts

{
  identifier: string,       // This is an unique identifier, `e.g. ur5e_gripper`.
  type: string,             // This is in our case `GenericSurfaceGripper`.
  prim_path: string,        // The prim path of the desired object in the scene, usually a surface gripper like tool
  robot: string,            // This is the identifier of the created robot, e.g. `ur5e`
  signals: string[],        // This is a list of signals, that means a list of controller IOs, e.g. ["tool_out[0]", "tool_out[1]"]
  translate: number,        // This is the offset in the direction you specified, default: 0.01
  direction: string,        // The direction of the surface, default: x
  grip_threshold: number,   // This defines the threshold limit for gripping an object, default: 0
  force_limit: number,      // This defines the limit under which force the grip lasts, default: 1000000
  torque_limit: number,     // This defines the limit for the torque of the surface gripper, default: 100000
  bend_angle: number,       // This is the angle of the surface bend, default: 0.1308996938995747
  kp: number,               // default: 100000
  kd: number,               // Damping value, default: 100000
  disable_gravity: boolean  // Gravity is only applied when activated. When deactivated and the object is released from the gripper, gravity is not applied, default: false
}

{
  identifier: string,    // Choose a name, e.g. ur5e_io_stream
  type: string,          // The type of the stream, in this case IOStateConnector
  robot: string,         // The identifier of your created robot, e.g. `ur5e`
}

Simultaneous configuration of robot and tools

When using the Omniservice API for configuration of the simulation environment, you can simultaneously configure the robot and tools. This allows for more complex custom configurations.

Use the endpoint Configuration > Load Configuration POST /configuration/load_configuration and use a .yaml file like the omniservice_config.yaml: This API endpoint is what's used in the Robot Pad so the required information might seem familiar: instance's IP address, the objects prim paths etc.

robots:
    0@ur10e:
        configuration:
            host: <my.instance.wandelbots.io>
            cell: cell
            controller_id: ur10e
            motion_group_id: 0
            prim_path: /World/cell/universalrobots_ur10e/universalrobots_ur10e
            is_secured: true
streams:
    0@ur10e_state_stream:
        configuration:
            type: RobotStateConnector
            robot: 0@ur10e
            robot_state: 50
    0@ur10e_io_stream:
        configuration:
            type: IOStateConnector
            robot: 0@ur10e
tools:
    gripper:
        configuration:
            type: GenericParallelGripper
            prim_path: /World/cell/universalrobots_ur10e/universalrobots_ur10e
            robot: 0@ur10e
            signals:
            - tool_out[0]
            - tool_out[1]
            open_position:
            - 0
            - 0
            close_position:
            - -0.03
            - 0.03
            default_DOF: 6
scene:
    uri:

Export and import configuration file

curl --request POST \
  --url http://localhost:8211/omniservice/configuration/export_configuration \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json'