Towards Physics-based Interactive Simulation of
Towards Physics-based Interactive Simulation of
Electrocautery Procedures using PhysX
Zhonghua Lu 1 2 Ganesh Sankaranarayanan 1 Dhanannjay Deo 1 Dingfang Chen 2 Suvranu De 1
1. Mechanical, Aerospace and Nuclear Engineering, Rensselaer Polytechnic Institute, Troy, NewYork, 12180, U.S.A
2. Intelligent Manufacture and Control Institution, Wuhan University of Technology, Wuhan, Hubei, 430063, P.R.C A BSTRACT
Haptic enabled virtual reality surgical simulators are increasingly replacing more traditional training tools in teaching hospitals. Development of these simulators may be greatly facilitated using physics libraries such as NVIDIA’s PhysX. While volumetric models of soft bodies may be easily generated and simulated using such engines, it is not straightforward to develop complex surgical tasks such as surgical cutting and hence novel algorithms are necessary. Electrocautery is a tissue cutting process used in surgery to burn away soft tissues by localized heating using a specialized probe. Unlike typical surgical cutting with sharp instruments, the electrocautery process depends upon the duration of the tool tissue contact and the rate of heat conduction. The simulation of electrocautery depends on understanding the physics of heat conduction as well as empirical measurements of temperature in the tissue. In this paper we report a physics-based paradigm for the simulation of electrocautery procedures that can directly work on volumetric objects. Based on the solution characteristics of the conduction equation and empirical observations using a thermal imaging camera, we manipulate only the tetrahedral mesh vertices that are inside a sphere of influence whose centre is located at the tip of the electrocautery tool and which expands as a function of time. A 3D orthogonal plane is used to split the tetrahedral mesh vertices along the three Cartesian directions. Examples are provided from a realistic surgical simulation environment.
K EYWORDS : electrocauterization simulation, date rearrange, PhysX engine, haptic rendering
I NDEX T ERMS : I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism Animation and Virtual Reality; H.5.2 [Information Interfaces and Presentation]: User Interfaces—Haptic I/O; 1 I NTRODUCTION
Minimally invasive surgery (MIS) is becoming the choice for many surgical procedures due to short hospitalization and quick recovery. However, performing these surgeries requires significant training to develop good hand-eye coordination and dexterity in tool manipulation. Haptics-enabled surgical simulators have been increasingly used for training surgeons for
such procedures. Electrocautery is one of the important surgical processes which is used to burn tissue using localized heating through a specialized tool heated by high frequency electric current. This procedure is also used to coagulate and minimize blood loss during surgery.
The development of a multimodal surgical simulation environment may be greatly facilitated if a physics engines such as NVIDIA’s PhysX is utilized [1]. PhysX, which can be accelerated using the physics processing unit (PPU) or CUDA-enabled GeForce graphics processing unit (GPU) – provides an optimized set of methods for physics-based simulation. In general, the PhysX framework supports real-time simulation of various physical entities such as rigid bodies, soft bodies, cloth, joints, fluids, springs, and characters, as well as collision detection and response algorithms with a focus on application to the gaming industry. We have pointed out in [1] how such engines might be used to develop 3D surgical simulation environments. However, implementation of complex surgical procedures such as electrocautery is not straightforward. To accomplish this, understanding of the physics of heat conduction in a 3D medium needs to be coupled with an algorithm for cutting 3D volumetric meshes.
In this paper, we propose an algorithm for cutting soft tissue as part of an electrocautery process in a surgical simulator. A volumetric tetrahedral model is used with position based dynamics (PBD) for simulation. During cutting, the vertices of the tetrahedra in the model closest to the tool tip are detected and the tetrahedra are split along a three dimensional cutting plane. The cutting area is controlled by a sphere whose diameter grows with time to simulate heat conduction. We have developed an experimental setup which has been used to measure temperature profiles using a thermal imaging camera. The algorithm has been successfully implemented in a surgical simulation environment with force feedback. 2 R ELATED W ORK
Cutting is a very common and frequent operation during surgery. Cutting simulation has been studied by many researchers. The main approach to cutting can be classified into two types. In the first type, the existing geometry is manipulated by either splitting the shared vertices or removing them altogether [1]. In the second type new geometry is created by remeshing along the cut region. The remeshing could be either connecting new vertices or subdividing existing elements. After progressive cutting, the number of newly generated elements might be prohibitively high using this method.
In [3] a cutting simulation using tetrahedral elements was proposed which used a hybrid approach that had the elements of both the approaches. Using a cutting plane, the tetrahedral model was cut in real-time. Their approach was to split the vertex nodes when the cutting plane is closest to it. Only in situations when this cannot be achieved, the edges of the tetrahedrons were cut, which required remeshing. This approach reduced the number of newly generated tetrahedra and avoided small or badly shaped elements.
luz4@https://www.wendangku.net/doc/ba743145.html, sankag@https://www.wendangku.net/doc/ba743145.html, deod@https://www.wendangku.net/doc/ba743145.html, dfchen@https://www.wendangku.net/doc/ba743145.html, des@https://www.wendangku.net/doc/ba743145.html,
515
IEEE Haptics Symposium 2010
25 - 26 March, Waltham, Massachusetts, USA 978-1-4244-6822-5/10/$26.00 ?2010 IEEE
3 M ETHODS
We have implemented a cutting algorithm for electrocautery procedures as part of a laparoscopic surgical simulator. 3.1
The physics of heat conduction in a 3D isotropic medium
The temperature distribution θin a 3D homogeneous isotropic body of density ‘ρ’, thermal conductivity ‘k ’ and specific heat ‘c ’ is governed by the following hyperbolic differential equation
2(,,)c k Q t
θ
t ρθθ?=?+?r (1) where Q t (,,)d θτr is the amount of heat generated per unit time in the element d τsituated at a point with position vector r . It is straightforward to show that if heat is liberated at a point with position vector a at the constant rate of Q , the temperature distribution as a function of position (r) and time (t ) is obtained by solving the above equation and may be expressed as
2(,)144c Q t erf k k ρθπ????????=????????????
?r a r r a t ?
(2) This expression clearly indicates that the temperature is a function of the radial distance of the source from the point of evaluation (i.e.,?r a ) and the solution is spherically symmetric. This
equation also provides an expression for the evolution of the level sets of temperature as a function of time. 3.2
Thermal imaging of temperature field during controlled electrocautery
An experimental setup using a Phantom Premium 1.0 haptic interface device has been developed to perform electrocautery experiments in a controlled manner and measure interaction forces. A Valleylab SSEL2 electrosurgery generator has been used to perform experiments. The electrocautery tool is attached to a Nano 17 force sensor from ATI industrial using a fixture (Figure 1), which, in turn, is affixed to the Phantom tool tip. The setup is
designed to move the electrocautery tool along pre-determined trajectories with control on the velocity of cut and depth of cut. The settings on the electrocautery generator are used to control the electric waveform applied to the tissue. For cutting a peak to peak voltage of 2800 V with sinusoidal waveform at 500 KHz is used which is modulated at 120% for 65% duty cycle, with nominal power of 375 W.
An InfraCAM? SD thermal imaging camera from FLIR Inc has been used to measure the temperature field at the tip of the electrocautery tool. This light weight camera has a temperature range of -5 ° C to 350 ° C, a field of view of 25×25 degrees and thermal sensitivity of 2% at 25 ° C. It has an image resolution of 120×120 pixels and a video frame rate of 30 fps (NTPC). Figure 2 shows a temperature contour measured using this camera at the tool tip while the tool is translated in an experiment involving a sample of fresh pig liver. As expected, due to surface effects and tool motion, the contours are not exactly spherical.
Figure 2. Temperature contours at the cautery tool tip 3.3 Deformable body simulation using PhysX
Soft body simulation is one of the most notable features of the PhysX framework in the context of this work. The framework simulates any given topological elastic body. In general, it differentiates the simulation mesh geometry from the mesh which is used for rendering the soft body. Therefore, the SDK uses a tetrahedral mesh for the internal structure, which is encapsulated by the surface mesh of the body. The method of integration used is semi-implicit. Verlet integration is applied to compute the new positions of the nodes of each tetrahedron. The updated positions of the vertices under external and internal forces (from springs connected to the vertices of the tetrahedra) are handled explicitly.
We have associated PhysX with haptic interface devices such as
the Sensable Omni? through our in-house software framework for interactive multimodal simulations (SoFMIS). Haptic devices provide positions and orientations of the users hands (Figure 3).
Force feedback Interact PhysX
C ollision with tool P ick up/drag/release
C utting
Haptic device Tool’s position Tool’s normal
Update Render Figure 3. Simulation overview
Rendering G raphic rendering Haptic rendering
Rearrange data
Select rendering data
Texture coordinates Calculate force
Figure 1. Experimental setup for electrocautery 516
50
100150200250300350
Tissue sample Phantom Premium 1.0Force Sensor Electrocautery Tool Fixture
PhysX uses the haptic device data or the cutting tool data as the constraint data to be applied to the soft body simulator in PhysX. Several procedures that must be performed by the tool include pick up, dragging of the tissue and release as well as cutting. After PhysX simulation, the data is rearranged from the PhysX data buffer to our format for real time graphic and haptic rendering. The main updates in this step are selecting the rendering data for tetrahedra some of which should disappear to simulate the effect of the electrocautery tool, updating the texture coordinates of the newly created nodes, and calculating the force feedback during the cutting operation along the cutting path. The culmination of this process is the graphical rendering of the images and haptic rendering of the tool tip forces to the user.
3.4 Electrocautery simulation using PhysX
When the electrocautery tool interacts with soft tissue, the information is provided by the collision detection algorithm. A sphere of constant temperature surface is generated at the tip of the electrocautery tool which evolves as a function of time (Figure 4). Nodes that are within the expanding sphere for a predetermined length of time are assumed to exceed the threshold and are split using three cutting places along the three Cartesian directions.
PhysX provides two methods to cut a tetrahedron; an implicit method and an explicit method [4]. The implicit method involves setting the tear factor greater than one in its description. The tetrahedra will split if the vertices are stretched beyond the tear factor. This method is totally automatic and we can’t handle which vertices are split. The explicit method, on the other hand, provides a means to control where and how the cutting progresses. We need two parameters: the vertex index and the cut plane, to define the tearing as Figure 5 shows. The vertex is duplicated first, and then the tetrahedron on one side of the cut plane retains the original vertex. All tetrahedra on the opposite side of the original vertex are replaced by the new one. The cutting plane is defined by the world location of the vertex and the normal is provided as a result of user interaction. The explicit method is therefore useful in electrocautery cutting simulation.
3.5 Graphical rendering
For soft body simulation, PhysX uses a tetrahedral volumetric mesh for the physics-based simulation but a surface mesh for graphical rendering. Each vertex of the surface mesh is linked to a certain tetrahedron of the underlying volumetric mesh, however, no topological relationship is provided between the surface mesh and the tetrahedral volume mesh, and it is very complicated to rebuild or change the topological structure of the surface mesh when the topology of tetrahedral volume mesh is changed during the cutting process. Hence a method has been developed to render the volume mesh directly without the surface mesh.
The PhysX raw data buffer is not suitable for real-time graphical rendering. So we need convert it to our data format. According to a predefined rule, we can derive the four triangle faces of each tetrahedron and is rendered in OpenGL.
In electrocautery simulation, the burnt tissue should disappear, which is taken care of by our rendering algorithm. We set a rendering state for every tetrahedron. If the state is ‘true’, the tetrahedron is rendered, otherwise the tetrahedron is isolated and it is not rendered. At the end of each simulation loop, we check the vertex data buffer. If the number of vertices hasn’t changed, no cutting has happened, and the volume mesh is rendered. If the number of vertices is larger than before, nodes have been split and new tetrahedral have been created. In that case, we check these tetrahedra using the indexes buffer. If they are isolated, we turn their rendering state to ‘false’, and render the rest of the tetrahedra. Texture mapping is applied to achieve visual realism which must also be updated in real time. The texture coordinate of each node is predefined at initialization. As the Figure 5 shows, the node is split into two nodes on either side of the cutting plane, one node is the original node and the other is the newly created one. The texture coordinate of the new node is initialized as the same as the original node.
Electrocautery Cutting Spheres
Figure 4. Cutting spheres 3.6 Haptic Rendering
Several haptic rendering techniques have been developed. In point-based haptic interactions, only the end point of the haptic device, also known as the haptic interface point (HIP) is taken into account in the collision detection between the objects. In ray-based haptic interactions, the haptic device is modelled as a finite line segment [5]. The point-based haptic interaction is the simplest paradigm, but it is unrealistic in some applications, while the ray-based paradigm is more natural but computationally more expensive.
In both point and ray-based force rendering, the point of the haptic device penetrates into the object, and the depth of penetration between the current haptic interface point and surface point is used to compute the interaction force. This is not possible in electrocautery simulation during which the mesh topology is changing as well as the surface point. Hence, we developed a different technique for computing the cutting force.
In PhysX, at the beginning of the simulation, the rest lengths of all the tetrahedral edges were recorded. During simulation, when the tip of the electrosurgery tool displaces the tetrahedra nodes, we compute the force as the difference between the rest length and the current stretched length of the edge and render that to the user. When the nodes are split for cutting, the force component corresponding to the connected edge would be removed resulting in a realistic effect of snapping of the tissue connections.
Cutting plane
Figure 5. Diagram for the cutting process 4 I MPLEMENTATION
We have implemented the electrocautery simulation in a Laparoscopic Adjustable Gastric Banding (LAGB) surgical simulator. LAGB is a complex minimally invasive surgical procedure which consists of placing a Lap Ban d around the stomach of a morbidly obese patient to constrict the passage of
517
food which results in early satiation. The one important step during the whole surgery is cutting the ligament between stomach and liver for the preparation of the band to go through as the Figure 6 shows. The electrocautery simulation during LAGB is showed in Figure 7. The tools were modeled based on real surgical tools in Autodesk 3DS Max software. The simulations were performed on an Intel Core 2 Quad Q9550 CPU machine with NVIDIA GTX 280 graphic cards. Two PHANToM Omni haptic interface device were used for force feedback. We observed that the frequency of PhysX update is around 30Hz, while graphic rendering is around 60Hz and haptic rendering is around 700 Hz.
Figure 6. Actual electrocautery in LAGB
Figure 7. Electrocautery simulation 5 C ONCLUDING R EMARKS
We have developed a paradigm of simulating electrocautery procedures for surgical simulation. The temperature iso-surface is approximated as an expanding sphere, sometimes known in the mathematics literature as the “heat ball”. However, actual experiments using a thermal imaging camera shows that the iso-surfaces are not exactly spherical due to effects of organ boundaries and motion of the tool. Besides, soft tissue is neither homogeneous, nor isotropic, both of which are assumed in equation (1). An alternative is to couple a fast solver for the heat conduction equation with the physics-based mechanical solver for tissue deformation. However, this is computationally expensive and is left as future work. We have also developed smoke generation and bleeding simulations which can be coupled with the electrocautery procedure for enhanced realism. Finally, it should be noted that the detailed electro-thermomechanics of the electrocautery procedure is highly complex and tissue dissection involves prediction of damage initiation and final rupture of the
tissue. A complete physics-based description of this process must be developed though further experimentation and modeling before it can be implemented in a surgical simulation framework. A CKNOWLEDGMENTS
This work is supported by No. R01 EB005807-01 from NIH/NIBIB. R EFERENCES
[1] Maciel A, Halic T, and Lu Z et al. Using the PhysX engine for
physics-based virtual surgery with force feedback. In Int J Med Robotics Comput Assist Surg 5: 341–353, 2009.
[2] Delingette H, Cotin S, and Ayache N. A hybrid elastic model
allowing real-time cutting, deformations and force-feedback for surgery training and simulation. In The Visual Computer , 16(8):437–452, 2000.
[3] Steinemann D, Harders M, and Gross M et al. Hybrid Cutting of
Deformable Solids. In Proceedings of the IEEE Virtual Reality Conference , 35-42,2006
[4] PhysX Introduction, In NVIDIA PhysX SDK 2.8 – Introduction,2008 [5] Basdogan C. Principles of Haptic Rendering for Virtual
Environments, online.
518