Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Haptic Perception with Artificial Tissues

Qualcomm Institute, University of California, San Diego, La Jolla, CA 92109, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1004632

Abstract

Haptic perception is critical in Minimally Invasive Surgeries (MIS) such as laparoscopic and robotic procedures in which the field of view is limited and the haptic force feedback is distorted or not available. Alternative haptic feedback is rendered by visual elements such as forced tissue deformation or using physical haptic interfaces with augmented reality. These approaches normally need additional training, add-on devices, and maintenance. In this study, we investigate an affordable method for creating a multimodal training simulator that integrates augmented reality (AR), extended reality (XR), and realistic artificial tissues from available human CT data. For the physical artificial tissues our objectives are threefold: first, to objectively measure tissue or organ hardness using a durometer in Shore Units (SU); second, to efficiently produce tissues and organs based on reference SU values and CT data; and third, to create specialized tissues. Additionally, we aim to arrange organs and tissues according to CT data, exemplified by forming the Calot Triangle for cholecystectomy surgery training. Finally, experienced surgeons tested the artificial tissues and organs inside the realistic cavity for basic surgical operation and provided professional feedback.

INTRODUCTION

Haptic perception, or tactile sensation, is an innate capability shared by humans, animals, insects, and even cells. In the context of Minimally Inva- sive Surgeries (MIS), such as laparoscopic and robotic surgeries, where the field of view is constrained, and haptic force feedback may be compromised or absent, haptic perception plays a crucial role. Traditional haptic feedback methods often rely on visual cues, involving forced tissue deformation or physical haptic interfaces within augmented reality frameworks. However, these approaches typically necessitate additional training, the integration of add-on devices, and ongoing maintenance.

Haptic perception represents a challenge in laparoscopic surgery, where the absence of tactile feedback can be a significant limitation. The ability to feel tissues is crucial for surgeons, especially in tasks such as identifying tumors and delineating their boundaries before excision. The current laparoscopy instruments struggle to provide effective haptic feedback (Eskef et al., 2011), contributing to the challenge. Furthermore, laparoscopic surgery introduces a dissonance between the visual and haptic systems, leading to a discordant perception that affects psychomotor output sequencing. This discrepancy necessitates a substantial period of compensatory adjustment, exemplified

© 2024. Published by AHFE Open Access. All rights reserved. 230

Haptic Perception With Artificial Tissues 231

by the perceived inversion of movement of the laparoscopic actuator’s tip— a phenomenon attributed to the “fulcrum effect” of the abdominal wall (Crothers et al., 1999). Studies have indicated that manipulating the laparo- scopic image, particularly inverting it around the vertical axis (y-axis), can expedite the learning process for novice subjects (Gallagher et al., 1998). This inversion can be seamlessly implemented in virtual reality (VR) training sys- tems, providing one potential avenue for addressing the challenges associated with haptic perception in laparoscopic surgery.

In this study, we explore an affordable way to produce realistic artificial tissues from available human CT data. We aim to solve three problems along the way.

First, objectively measure the hardness of tissues or organs with a durom- eter for the Shore Unit (SU); Second, rapidly produce the tissues or organs based on the reference SU values; Third, make special tissues such as cys- tic duct and cystic artery that are much smaller and hollow; Four, arrange the organs and tissues according to CT data, for example, forming the Calot Triangle for cholecystectomy surgery training; Finally, experienced surgeons tested the artificial tissues and organs inside the realistic cavity for basic surgical operation and provided professional feedback.

HAPTIC INTERFACES The laparoscopic surgery training landscape encompasses a spectrum of sim- ulators, ranging from basic laparoscopic surgery phantoms to advanced VR digital simulators and high-end da Vinci robotic surgery system simulators. Research indicates that novice individuals can rapidly acquire proficiency in straightforward tasks like clipping and cutting commonly seen in current basic training simulators. Despite the advancements in laparoscopic surgery training, current systems commonly fall short in adequately simulating haptic perception, providing real-time performance assessment and feedback, and offering realistic scenarios for surgical procedure training. Furthermore, the prevalent issue of the proprietary and costly nature of many existing systems acts as a barrier, limiting the widespread adoption of more advanced train- ing platforms. The need for more inclusive, cost-effective, and comprehensive training solutions remains a pivotal challenge in the field.

The haptic interface in our system simulates tactile feedback from tissues, liquids, and various materials. In Virtual Reality (VR) mode, this interface conveys force feedback to the operator at the end of the laparoscope, gen- erating tactile sensations through physical simulation models. These models encompass actions such as tearing, sewing, grabbing, touching, stapling, and more. On the other hand, Augmented Reality (AR) mode involves the use of physical laparoscope kits and 3D-printed silicone tissues or silicone cast from 3D-printed molds for practical exercises. In this study, we focus on developing an economical haptic interface capable of sensing the position and orientation of each figure, providing corresponding force feedback. Simultaneously, we are working on AR algo- rithms to overlay photorealistic visual details, such as blood and tissues, onto the silicone models, enhancing the overall training experience.

232 Cai and Perez

For the design and simulation of haptic interfaces, haptic engineering soft- ware options such as Chai3D and OpenHaptics are available. These tools facilitate the development and testing of haptic feedback mechanisms. Addi- tionally, commercially off-the-shelf controllers and Head-Mounted Displays (HMDs) prove to be both affordable and accurate for the Extended Reality (XR) simulator, particularly in terms of tracking the position and orienta- tion of laparoscopes. According to the study of the VR controller Oculus Touch (Rojo, 2022), the maximum positional accuracy error of the Ocu- lus Touch was 3.5±2.53 mm at the largest step size of 500 mm along the z-axis and the rotational accuracy of the system was 0.34o±0.38o for the HMD and 1.13o±1.23o for the controller. Compared to the high-end laparo- scopic surgery systems da Vinci and MAKO robot across three types of surgeries, accuracy in robotic-assisted spinal surgery reported 1.5 to 6.0 mm of translation and 1.5

  • to 5.0
  • of rotation when comparing planned to final implant position (Cunningham et al., 2021). So the accuracy of position and orientation tracking is within the acceptable range.

The haptic interface Phantom has been used in many high-end medical simulators including da Vinci robotic surgery simulators. Renowned for its high accuracy, it remains a key player in haptic devices. However, cer- tain limitations, such as the absence of grasping force feedback present in laparoscopic surgeries and its high cost, have prompted the exploration of alternative solutions. In this project, the goal is to integrate a simplified Phan- tom with a handheld sensor and haptic device, specifically Oculus Touch. These components are linked by a rigid rod that emulates the motion of a laparoscope. The simplified Phantom, designed to be 3D printed, addresses the need for a more affordable solution, offering a cost-effective alternative without compromising on the essential aspects of haptic feedback and motion simulation.

An evaluation was carried out to assess the quality of haptic feedback, comparing the Phantom Desktop, Phantom Omni, Novint Falcon, and a simplified Phantom kit referred to as “Woody.” The perceived quality was evaluated through a questionnaire, utilizing a seven-point Likert-type scale. Respondents were prompted to rate to what extent the haptic feedback was deemed to be of high quality, precise, smooth, and distinct. This assessment aimed to capture the experienced nuances of haptic feedback across these dif- ferent systems. The results, as the average summed ratings (std. dev.) of each device are as follows: Desktop Phantom 25.8 (2.10), Omni 19.5 (5.7), Falcon 11.2 (6.3) and Woody 24.4 (3.37). The users rated the simplified Phantom’s experience between Desktop Phantom and Phantom Omni (Forsslund et al., 2015). Additionally, the investigation extends to alternative haptic interfaces, considering factors such as accuracy and cost. Examples include the Reflec- tive Grip™from Tactical Haptics (Provancher et al., 2019) and various do-it-yourself (DIY) haptic designs (Hayward and Macclean, 2007).

In the Augmented Reality (AR) mode of the training simulator, trainees will utilize physical laparoscope kits along with 3D-printed organic silicone tissues for practical exercises. The versatility of silicones is notable, as they can exhibit fluid, viscous, pasty, elastomeric, or rigid textures. Historically, producing silicone parts has been prohibitively expensive due to the high costs

Haptic Perception With Artificial Tissues 233

associated with molds. A significant breakthrough in this regard is the advent of the “drop-on-demand” process, enabling room-temperature 3D printing with silicone (Beamler, 2023).

For this project, we are collaborating with Wacker’s ACEO (Wacker, 2018) and Elkem 3D printer silicon products (Elkem, 2019) to facilitate cost- effective and rapid production. The focus includes developing Augmented Reality (AR) algorithms to overlay photorealistic visual details, such as blood and tissues, onto the silicone models. We are concurrently working on the creation of a Universal Laparoscopic Scene Description (ULSD), serving as a digital library comprising downloadable 3D abdomen tissue models. These models can be 3D printed, utilized to create silicone molds, and subsequently cast to produce realistic silicone tissues with varied textures, colors, and even vessels. This approach aims to streamline and enhance the production of lifelike training materials for the laparoscopic surgery simulator.

TISSUE HARDNESS MEASUREMENT Except for bones, human tissues generally exhibit a soft consistency. Internal tissues, characterized by varying hardnesses and weights, manifest a multi- tude of layers and diverse characteristics. Notably, these tissues vary in size and physical properties. In this study, we systematically compiled existing data on tissue or organ hardness measured in Shore Units (SU) through the utilization of a durometer. To evaluate the hardness of our produced tis- sues and organs, we employed a Shore 00 hardness scale durometer. This allowed for a direct comparison between the Shore Unit values obtained from our simulated tissues and organs with those acquired from actual spec- imens. As of now, research involving the application of the Shore hardness scale to real human tissues is limited. The conclusive data available pertains specifically to the Human Liver, Pancreas, and Kidney, which are integral components of the tissues under development for our haptic abdominal sim- ulator. Table 1 presents the measured objective values obtained from this comparative analysis.

Table 1. Shore unit (SU) values of healthy targeted organs and their replicas.

Table available in the original PDF paper.

Organ Targeted Shore Unit (SU) Tested Shore Unit (SU) Liver 15.06 ± 2.64 Shore 00 (Yoon et al., 2017) 17.20 ± 2.00 Shore 00 Pancreas 26.3 ± 2.5 Shore 00 (Foitzik et al., 2006) 25.20 ± 2.00 Shore 00 Kidney 36 ± 10 Shore 00 (Tejo-Otero et al., 2022) 37.50 ± 2.00 Shore 00

Table 2. Sample shore unit (SU) values of replicated healthy organs and tissues.

Table available in the original PDF paper.

Organ Tested Shore Unit (SU) Gallbladder 19.3 ± 2.00 Shore 00 Small Intestine 22.00 ± 2.00 Shore 00 Stomach 26.80 ± 2.00 Shore 00 Colon 25.50 ± 2.00 Shore 00 Spleen 17.50 ± 2.00 Shore 00

234 Cai and Perez

In the absence of conclusive data on Shore Unit Hardness for various other abdominal organs, we relied on iterative feedback from surgeons to fabricate tissue replicas that closely mimic the mechanical properties and haptic feed- back of real human organs. Table 2 presents the Shore Unit hardness values obtained through testing our current tissue replicas, where a specific target Shore hardness was not established.

The data pertains exclusively to normal, healthy human tissues. It is impor- tant to note that the hardness of tissues undergoes alterations in the presence of complications or pathological conditions. For instance, the gallbladder experiences a significant increase in hardness when afflicted by gallstones. Consequently, it becomes imperative to devise distinct models with varying Shore Unit hardness values to accurately simulate unhealthy organs or tissues in our study.

FABRICATION OF ARTIFICIAL TISSUES Various methods exist for fabricating physical tissues or organs, ranging from biological cell cultivation to advanced 3D printing. In the context of developing a laparoscopic surgery training simulator, we have explored a straightforward, cost-effective, and realistic approach—casting.

Initially, digital 3D models are generated from CT DICOM data to accu- rately replicate the volume and shape of organs and tissues. Using 3D Slicer software, digestive organs such as the liver, gallbladder, kidneys, spleen, stom- ach, colon, and small intestine can be individually extracted from the CT data. Subsequently, these 3D models are produced at a 1:1 scale using a 3D printer.

Upon obtaining a physical 3D-printed model of the target organ, silicone is cast around it to create a reusable negative mold of the desired object. This mold serves as a foundation for casting replica organs that closely emulate the mechanical and haptic feedback properties of the targeted organ or tissue. Different materials and pigments are employed for distinct organs and tissues, ensuring an accurate representation of their specific haptic, visual, and inter- action properties. Special glues and silicone are then utilized for connecting various organs and tissues seamlessly.

The fabrication of hollow organs and vessels poses a significant chal- lenge, particularly with conventional molding methods that entail multiple casting parts and intricate processes, rendering the overall procedure com- plex and time-consuming. In our pursuit of innovative solutions for crafting hollow organs and vessels, such as those found in tubular tissues such as small intestines and arteries, we acknowledge the inherent difficulties. While progress has been made in overcoming challenges for smaller tubular struc- tures, scaling up to larger organs, such as the colon, remains a complex task.

In addition, we employed 3D printing to create a simulated abdominal cav- ity for laparoscopic surgery training. The design of the abdominal cavity is tailored to accommodate the digestive organs and tissues. Utilizing the same CT data from which the organ models were derived, we crafted a 3D model of the abdominal cavity. This cavity comprises separable top and bottom parts,

Haptic Perception With Artificial Tissues 235

each characterized by distinct hardness. The top part emulates the skin using a soft material, allowing laparoscopic trocars to penetrate the surface dur- ing port location training. Figure 1 illustrates the physical digestive system, featuring artificial tissues within the abdominal cavity.

Figure 1: The physical digestive system with artificial tissues in the abdominal cavity.
Original PDF Figure 1:

ANATOMIC VARIATIONS Numerous surgical injuries arise from variations in anatomic structures due to the inherent diversity in biological characteristics, encompassing size, shape, location, connection, layout, color, texture, hardness, weight, angle, and more. Taking the gallbladder vessels as an example, there exist a mini- mum of eight distinct types of variations, as illustrated in Figure 2. To address these variations, we are utilizing a method where the gallbladder remains consistent while the duct variations are reconfigured using soft hollow tis- sues affixed at specific locations. Geometric variations can be reproduced through printing on paper or carving on molds for efficient mass produc- tion in modular units. Additionally, anatomic variations can be introduced through digital object overlays; however, it is essential to note that overlaid virtual objects alone do not provide the necessary haptic force feedback.

Figure 2: The artificial tissues can be arranged for variations of the gallbladder vessels.
Original PDF Figure 2:

LAPAROSCOPIC SURGERY SIMULATION In the construction of our laparoscopic surgery simulator, we seamlessly integrate the laparoscopic camera and laparoscopes into the digestive cavity

236 Cai and Perez

model. The laparoscopic camera, connected to the computer via a USB port, facilitates live video streaming during simulations. This setup enhances real- ism, affordability, and intuitiveness compared to alternative haptic interfaces like the haptic arm Phantom and modified virtual reality game controllers.

Beyond the physical simulation involving artificial organs and tissues, our simulator incorporates Extended Reality (XR) components. This includes the detection and tracking of laparoscopic instruments and camera views, allow- ing for comprehensive assessments of camera holding performance. Many surgical scenarios involve an assistant holding the camera, positioning it in alignment with the laparoscopic instrument, and centering it on the screen. Additionally, surgeons can leverage XR components to detect and track anatomical regions, particularly critical areas such as Calot’s Triangle. This triangle, formed by the bottom of the liver, cystic duct, and cystic artery, is a region where identifying and tracking structures is crucial. Figure 3 depicts the physical digestive system within the abdominal cavity, featuring artifi- cial tissues for laparoscopic surgery training. The overlaid blue bounding box highlights the Calot Triangle region, showcasing the potential of XR components in enhancing surgical education.

We tested this system with 20–25 medical residents and medical students at Goodman Surgical Education Center at Stanford University.

Figure 3: The abdominal cavity with artificial tissues for laparoscopic surgery training. The overlaid blue bounding box is the Calot Triangle region.
Original PDF Figure 3:

CONCLUSION

In conclusion, this paper navigates the complex terrain of haptic perception in the context of Minimally Invasive Surgeries (MIS), with a specific emphasis on its pivotal role in laparoscopic procedures. The challenges stemming from the absence of tactile feedback in laparoscopy, coupled with the discordance between visual and haptic systems, emphasize the urgent need for inventive solutions.

Haptic Perception With Artificial Tissues 237

This study introduces an innovative approach to address these challenges, presenting an economical haptic interface that harnesses 3D printing and sil- icone casting to fabricate lifelike artificial tissues for laparoscopic surgery training. These artificial tissues not only prove to have realistic tactile prop- erties but are also cost-effective. The integration of Augmented Reality (AR) further enriches the training simulator, immersing trainees in a realistic visual environment. The development of Extended Reality (XR) components, encompassing laparoscopic instruments and camera tracking, expands the simulation’s capabilities beyond physical models.

In essence, the presented approach not only addresses the complexities of haptic perception in laparoscopic surgery but also advocates for a multi- modal training paradigm. By seamlessly incorporating cost-effective haptic interfaces, realistic artificial tissues, and cutting-edge XR components, this research paves the way for redefining the landscape of laparoscopic surgery education. The positive feedback from medical professionals signals the potential transformative impact of this work on advancing surgical training methodologies and, ultimately, improving patient outcomes.

ACKNOWLEDGMENT

The authors would like to thank the support of the Wellcome Leap SAVE Program and the NIST PSCR Program.

REFERENCES

Beamler (2023) Is 3D printing silicone possible? https://www.beamler.com/is-3dprinting-with-silicone-possible/ Crothers, I. R. Gallagher, A. G. McClure, N. James, D. T. D. and McGuigan, J. (1999).

Experienced laparoscopic surgeons are automated to the “fulcrum effect”: An ergonomic demonstration. Endoscopy, vol. 31, no. 5, pp. 365–369, 1999. Cunningham, B. W. Brooks, D. M. McAfee, P. C. (2021). Accuracy of Robotic- Assisted Spinal Surgery-Comparison to TJR Robotics, da Vinci Robotics, and Optoelectronic Laboratory Robotics. Int J Spine Surg. 2021 Oct;15(s2): S38-S55. doi: 10.14444/8139. Epub 2021 Oct 4. PMID: 34607917; PMCID: PMC8532535. Elkem (2019). Silicone for 3D Printing and Additive Manufacturing https://www.elkem.com/silicones/offer/additive-manufacturing/ Eskef K, Oehmke F, Tchartchian G, Muenstedt K, Tinneberg HR, Hackethal A. A new

variable- view rigid endoscope evaluated in advanced gynecologic laparoscopy: A pilot study. Surg Endosc 2011;25(10). Foitzik, T., Gock, M., Schramm, C. et al. Octreotide hardens the pancreas. Langenbecks Arch Surg 391, 108–112 (2006). https://doi.org/10.1007/s00423-006- 0030-z Forsslund, Jonas & Yip, Michael & Sallnäs, Eva-Lotta. (2015). WoodenHap-

tics: A Starting Kit for Crafting Force-Reflecting Spatial Haptic Devices. TEI 2015 - Proceedings of the 9th International Conference on Tangible, Embed- ded, and Embodied Interaction. 10.1145/2677199.2680595. Gallagher, A. G. McClure, N. McGuigan, J. Ritchie, K. and Sheehy, N. P. (1998). An ergonomic analysis of the fulcrum effect in the acquisition of endoscopic skills. Endoscopy, vol. 30, no. 7, pp. 617–620, 1998.

238 Cai and Perez

Hayward, V. and Maclean, K. (2007). Do it Yourself Haptics: Part 1 [Tutorial], IEEE Robotics and Automation Magazine 14(99), Dec. 2007. Provancher, W. R. (2019). Creating Fully Immersive Virtual and Augmented Reality

by Emulating Force Feedback with Reactive Grip™Touch Feedback. Revised Aug. 2019. https://tacticalhaptics.com/files/TacticalHaptics_whitepaper_Aug2019-Rv 2.pdf Rojo, A. Cortina, J. Sánchez, C. et al. (2022). Accuracy study of the Oculus Touch v2 versus inertial sensor for a single-axis rotation simulating the elbow’s range of motion. Virtual Reality 26, 1651–1662 (2022). https://doi.org/10.1007/s10055- 022-00660-4 Tejo-Otero A, Fenollosa-Artés F, Achaerandio I, Rey-Vinolas S, Buj-Corral I, Mateos-

Timoneda MÁ, Engel E. Soft-Tissue-Mimicking Using Hydrogels for the Develop- ment of Phantoms. Gels. 2022 Jan 6;8(1):40. doi: 10.3390/gels8010040. PMID: 35049575; PMCID: PMC8774477. Wacker (2018) 3D printing lab for silicone opens in the US. https://www.wacker.com/ cms/en-us/online-magazin/detail-103849.html Yoon YC, Lee JS, Park SU, Kwon JH, Hong TH, Kim DG. Quantitative assessment

of liver fibrosis using shore durometer. Ann Surg Treat Res. 2017 Dec;93(6): 300–304. https://doi.org/10.4174/astr.2017.93.6.300

Cover of the proceedings volume Accessibility, Assistive Technology and Digital Environments

Explore the complete digital volume

Accessibility, Assistive Technology and Digital Environments

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-97-1

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →