Ray-casting inspired visualisation pipeline for multi-scale heterogeneous objects

Abstract

In this work, we consider a problem of visual analysis of dynamic multi-scale heterogeneous objects. For this task, we propose a GPU-accelerated visualisation pipeline with design principles being inspired by ray-casting. We explore the advantages of the object-oriented approach to data-flow visual pipeline design and show how this approach can be used to construct a GPU-accelerated visualisation pipeline. We discuss an example of the framework that is built on top of the Optix Engine and show how various multi-scale and heterogeneous objects can be visualised with the designed framework.

The source code is available at github page.

Problem overview

Visualisation frameworks allow designing a pipeline from the input data to the rendered image, as a set of modules of specified functional groups that deal with data filtering, mapping, rendering, etc. [Mor13]. Traditionally, the visualisation deals with modelling procedures that define the geometry and materials of the object’s surface. However, when it comes to volume objects, the only method allowing for user customisation is the transfer function [CJS*18]. In heterogeneous objects, the regions of non-homogeneous material distribution are defined within a set of geometric domains [KWN*14],[WKME03]. Such representation demands flexibility in modelling methods of geometry and materials, especially where dynamic multi-scale features are considered. On the other hand, we need to address the efficiency issues as the pipeline evaluation time grows with the number of basic procedures [Mor13]. GPU hardware and algorithms have changed significantly since most of the conventional frameworks have been designed [SLM06]. Techniques, such as ray-casting, take advantage of GPU programming flexibility [CJS*18], acceleration structures [BHP15] and on-demand loading of various data representations [RV06],[BHP15]. Moreover, the modern ray-casting techniques simplify the conventional geometric problems in visual pipeline design[KWN*14], [PV12].

In this work, we use the accelerated GPU ray-casting as inspiration and core of parallel visual pipeline design. We extend the technique with GPU-based definition of materials and geometry and take advantage of modern GPU-accelerated programmable features to enable GPU data flow.

Proposed approach

Let us consider a heterogeneous object with a material distribution defined within cellular space partitioning. Each cell is represented by a tuple \(\{G_i, A_{oi}\}\), where \(G_i\) is geometry and \(A_{oi}\) is a corresponding material. Volume ray-casting of a heterogeneous object is accelerated with hierarchical structures such as Bounding volume hierarchy (BVH) with the following steps:

  1. BVH is used to find all basic geometry primitives \(G_j\) that ray intersects. For example in the case of tetrahedral meshes, such primitives are tetrahedra [WFKH07]
  2. We assume that all primitives \(G_j\) are further sorted along the ray. The exact ray-primitive intersection values can be computed before or after the procedure.
  3. For each of the basic primitives \(G_j\) we use subvolume integration to render the material contribution. Conventional emission and absorption model for light-object interaction is used [JSYR14].

Most of the recent works in the area are limited in the way they use materials composed on convex subdomains such as tetrahedral volume meshes [WFKH07]or molecular point-based data [KWN*14]. In this work, we consider geometry represented with a signed distance field (SDF). The SDF framework provides a unified basis for geometry and material modelling on the GPU with convenient ray-casting. Our approach is outlined in Fig. 1. The geometry and material are modelled through functional superpositions [SPAS01]. The structure can be schematically presented in the form of a graph that is often referenced as a constructive tree [SPS08](see Fig. 1 ). In this work, the constructive trees map to the nodes of the visual pipeline’s modular logic. With dynamic multi-scale features a number of mappings can grow rapidly. As a consequence, a constructive tree grows and its evaluation time slows down the rendering[GPB*09],[Ngu06].

The modern GPU programming allows using custom programming procedures to address both modelling and efficiency aspects within visual pipeline design. In most cases, we can optimise the constructive tree evaluation by “splitting” tree into local subtrees with bounding subvolumes and loading necessary subtree on demand (see Fig. 1 ). Moreover, different constructive trees that represent multi-scale dynamic features can be loaded on demand due to parameters \(t={t_1,..,t_n}\) values in \((G_i(t),A_i(t))\) (see Fig. 2).

The introduced subdivision of bounding volumes and subvolumes allows taking into account both geometric model and material distribution. Very often there is a correlation in the material geometry regions and geometry used in SDF constructive tree and that simplifies the splitting task (see Fig. 1). Moreover, we can address even difficult, non-standard cases of complex geometry and topology dynamic changes when the constructive tree splitting is difficult. For such cases, a less efficient but more flexible bounding volume in a form of more simple dynamic SDF can be introduced.

Implementation

This work was inspired by the object-oriented principles in data-flow design that allow building a visual pipeline from basic building blocks in VTK framework [SLM06]. In this work, we use the possibilities of default and callable programs introduced in Optix to imitate the object-oriented tricks for GPU-based pipeline construction. Despite that the callable programs are convenient for GPU implementation of constructive modelling with SDF, the callable structure can significantly slow down the rendering of large heterogeneous objects. We compile the necessary CPU blocks by overwriting the Update procedure (similar to VTK concept) in classes instead of operating callable programs ids. The conventional visualisation tools specify the dimension of geometry and topology described in the input data. In this work, we deal with 0D case (points), 1D case( points and connecting line segments), 3D case (volumes). For those data types, we design of basic class hierarchy that in most cases suggest overriding one callable program that defines an SDF primitive. Additionally, we define a material callable program that depends on SDF program ID and in some cases is send to Renderer class, so it can perform volume sampling after primitives sorting. Here is an example of pipeline implementation to give a general idea on the programming structure of provided visualisation examples. It is quite simple and very similar to VTK.

Conclusion

The features and acceleration structures of GPU ray-casting can be used as a core of heterogeneous object visual pipeline design. The presented research is a work in progress. The presented in project ideas are inspired by the requirements imposed in the area of digital fabrication and real-time user interaction with multisensory feedback. Future work aims dealing with more cases interactive cases that consider material and geometry modelling. The source code with examples can be found at github page. The main target is to provide a demonstration examples of introduced approach flexibility and user friendliness. However, the source code is a quickly extracted part of a bigger project that currently in a state of active development and very draft codes does present.

References

, p. .