Introduction to udSDK in Unity

Today we will be exploring the features of our unity plugin. By the end of this post we will be able to load point clouds within the Unity editor, design applications using unity, and use some of the integrated features within Unity including

  • Pointcloud picking
  • Filtering of pointclouds
  • Real time generation of local mesh colliders for our point cloud

Our Unity example is a wrapper for the C language udSDK API written for Unity. It includes C# bindings for a subset of features of the udSDK API as well as Unity Monobehaviour scripts and demonstrations of their use in our demonstration projects. It does not currently have interfaces to all available features of the udSDK API; for a full list of available functions consult the udSDK documentation included with your udSDK download.

Additional content for this plugin can be requested and we love to hear what you are doing with it.

This tutorial assumes a basic familiarity with the Unity editor.

Getting Started

udSDK Unity can be obtained from GitHub at https://github.com/Euclideon/vaultsdkunity. Installation instruction are available through the readme.md

Logging In

Starting Up

Once the package has been installed, start by opening Scenes/Basic Render.

Building the Package

Make sure to set this to Scene 0 in the settings before building:

For your end user to sign into the udStream Server we have provided a simple login screen scene (Menu 3D) that will facilitate this. This is only activated in the case that we are not using the editor.

The Components

A number of examples are predefined in Assets/prefabs:

The Collider

Included with the sample is an example of using the VDK renderer for the generation of local meshes for the purpose of collision detection. This is used in our samples in two ways: to determine collision in physics-based simulations and games, and ray casting into the scene.

Configuration and customization of this feature is very dependent on use case and warrants another post of its own. You can refer to the included documentation in the readme.md included with the git download for additional technical information on how to configure it.

Ray casting example

In addition to our example of collision detection, we also provide an example of ray casting into the scene by using the above collider to estimate the surface we are casting to. This is used to calculate the distance from the camera to the surface and log that distance. This is not the recommended method of determining distances between point in the cloud as the picking system provides a more accurate and direct method of doing so. The ray casting example can be accessed from the basic example scene, after running the sample in the editor, we can activate it by pressing space. The ray can be seen projecting from the main camera to the surface

The Driving demo

An additional scene demonstrating practical usage of the collider system for a driving simulator has been included. It can be found in the project under Scenes/Driving Demo

This shows how the Unity driving simulator example can be integrated with point cloud models using the vdkCollider System to generate a mesh local to the model.

The Picking System

VDK provides an interface for extraction of voxel data based on screen space selection of the corresponding pixel. The basic idea here is we can select a pixel on our screen and the picking system will return information about the points we selected. These include the position in space (x,y,z) of the voxel (useful if we want to know the distance between different points in our scene).

We can also use the voxel ID returned by this process to get any additional stored information about the voxel that is stored in the UDS (e.g. classification, displacement data, material data, etc.). This is an advanced feature and while our Unity API wrapper does not currently support this all information is available to support this for your projects. Support is available for particular applications of this feature on request, and general support for this may be supported in future releases.

An example of how to use this interface is in the basic render Demo. We can enable this feature by selecting “Place Cubes on Pick” in the VDK Camera options of the camera object: this will allow us to spawn a cube at the location in the point cloud that we have clicked on, “show pick marker” will show a cube at the location our cursor is on.

 Our sample allows the user to click on a point in space and place a cube at the location they have picked. The coordinates in world space of this point are also logged to the debug console. Double clicking this message when picking will open a location appropriate to put callbacks when using the picking  interface in your own projects.

Query Filter

The query filter is an object for selecting points from your cloud. Currently we demonstrate the ability to isolate or exclude parts of the model for viewing. The model is placed in the scene as a box which can be scaled, rotated and translated using the standard Unity transform tools

Setting the inverted Boolean to true will result in the contents of the filter being excluded and the rest of the model being rendered:

Currently only one of these filters can be active at a time; the behaviour for multiple active filters in a scene is undefined, though the observed behaviour is that the last placed filter will be the one which is rendered.

It also allows you to export parts of the model as points for analysis or conversion. Note that this process is quite computationally expensive and is intended for offline processing and export of UDS files to other formats. For support in implementing point export and analysis in Unity [contact us] for a customized solution. Currently we do not provide an interface in Unity to perform this, though the tools to do so are available in the udSDK library.

An example of the usage of this feature is contained in Scenes/Filter Demo

Conclusion

This post has covered a few of the possible features of udSDK when used in the Unity development environment. We have shown how to render, position and extract individual point positions, as well as filter model renders and approximate local geometry for physics simulation.