Fundamental VR: Using haptic feedback to create a VR surgical system

Compilation/VR Gyro Wonton

VR has always been used to create a whole new world, and now more and more people have discovered that VR has great potential in teaching, especially teaching that is more difficult to implement in the real world.

For example, using VR for training the next generation of doctors is a good example. Case Western Reserve University announced that it plans to bring VR into the school's anatomy laboratory for teaching corpses and use Microsoft HoloLens to teach. Interns who wear HoloLens can see different levels of the 3D human body - skin, muscles, blood vessels, and so on.

And a British company decided to go further in this area. The company, Fundamental VR, decided to reproduce the actual surgical procedure in VR and let the medical students feel the tactile sensation during the operation through tactile feedback.

Fundamental VR hopes that in this way, medical students can avoid practicing on real patients and reduce risks until their skills are mature.

The system developed by the company includes a HoloLens head and a stylus attached to a standard robot arm. In VR, the stylus appears to be a syringe, one button can empty the syringe, and the other button can refill the liquid.

Moving the stylus in the real world can move the syringe in VR. When virtual needles come into contact with virtual skin, muscles or bones, different resistances caused by different media can be transmitted to the user through the stylus, giving them realistic feelings about the real human body.

The basic idea of ​​this system is that the different parts of the human body, like fat and bones, should feel different.

The first system of Fundamental VR is used in knee arthroscopic surgery. The system was created specifically for pharmaceutical company Pacera, allowing clinicians to learn how to handle Exparel, a company's newly developed anesthetic.

Unlike many conventional anesthetics, Exparel does not spread to the body after one injection, but requires multiple injections at different sites and the drug will accumulate at the site of injection. For some surgeons, changes in the surgical procedure are difficult to master, so VR teaching came into being.

Through multi-angle imaging of the knee joint, the company has created a complete 3D model. Off-the-shelf haptic devices are used to replace syringes in the system, but of course any needed tools can be replaced as needed.

Full participation of doctors

Creating a system that truly reproduces the surgical experience requires a combination of human intelligence and high technology. When building VR knee replacement surgery, Fundamental VR consulted surgeons at every step.

“Surgery is a scientific process, but it is also an art. Where there is an art, there will be a different point of view. If we want to create a standard that most people agree with is very time-consuming. In the process, we have Prepare for the challenges, such as how to reproduce the texture and blood vessels of different muscles, and how these parts should change during the operation.” Richard Vincent, co-founder of Fundamental VR, said in an interview.

Next, the surgeons also participate in how to convert the actual surgical experience into a VR version. This requires the use of the company's tactile feedback development engine to connect the real and VR worlds.

“We built a calibration tool, which is the core of our software. It allows us to quickly and communicate several difficult points. At the beginning, our experimenters will say 'feel like a needle inserted an orange,' or' Like a chicken, after a few adjustments, most people would agree with the feeling in real surgery,” said Vincent.

Stan Dysart is a specialist in joint replacement surgery. He participated in the process of reconstructing the feeling of surgery in VR. He will assign a number to different elements of the human body to correspond to a tactile texture.

“The haptic device has a scoring system, and I can help developers determine what the needle feels in the joint capsule, how it feels in muscle, fat, in the periosteum, and bone,” he said. For example, he said that if it is in the fiber capsule, inserting the needle feels like inserting plastic.

"The joint capsule has a certain resistance. This resistance will be released as the needle goes deeper. In VR we can simulate this tactile change, and the touch in different parts of the knee is not the same. In these different senses After the numbering, the computer will be able to handle it accordingly - the higher the number, the greater the resistance the doctor will encounter, Dysart said.

When a team of surgeons agrees on the tactile sensations of different parts of the human body in the VR, the haptic system translates the number into the VR user's experience when using a virtual syringe—a process that also requires testing of the GPU's ability to handle multiple scenes.

According to Dysart, this virtual knee surgery system has been used at several medical centers in the United States. The system helps doctors continue to refine their skills.

“Surgeons like this way because they can practice continuously without harming patients. This is the most important point. In real surgery, we need to consider the consequences of every scalpel we move—this How deep is the cut? Where is the cut and where is it injected? Because the knee is full of nerves and arteries.

"In VR, if you accidentally pin the needle too deep, it doesn't matter. The doctor will know that you are not doing it right, and then practice it over and over again to get the right technique. This is the best part of this system."

In addition to knee surgery, Fundamental VR is currently developing another three surgical procedures, including soft tissue surgery and spinal surgery, with the opportunity to go online this summer.

Ideal teaching tool

Fundamental VR has been negotiating with many medical education institutions and hopes to use the haptic feedback VR system to teach medical students to improve surgical skills or to allow doctors to learn new surgical therapies. At present, Fundamental VR focuses on the U.S. market and is also establishing links with medical institutions in the United Kingdom and other countries.

In addition to tailoring VR systems for customers, the company hopes to establish a collection of common surgical procedures in the future. Through subscriptions, doctors around the world will have access to these materials. In addition to routine surgery, the company also exposes medical students to relatively rare anatomical cases and surgical environments.

“This is one of the most common operations, such as appendectomy, and even if it is not demonstrated by VR, medical students will have many opportunities to observe the operation. However, neurological surgery is different, and some operations may be one year.” None of them will be encountered three times, but these surgeries are usually very important and life-and-death. There are not many doctors who need these surgeries, so it is a good idea to familiarize them with VR."

The haptic system is not suitable for all doctors - some experienced doctors will think that the VR simulation scene is too much like a computer game, but Fundamental VR thinks this is the same as letting the pilot accept the flight simulation.

“We talked to many surgeons at various exhibitions and they encountered many urgent situations in real surgery, and usually only a few minutes to make decisions. If we train through our system beforehand, In any case, it will be more easily handled in the event of an emergency,” explains Vincent.

Practice in VR, not the patient

When MR and VR become more affordable and ubiquitous, specific models can be created using tactile and VR before the patient begins surgery.

“Before surgery, if we can discuss in advance on these models how to deal with organs, how to move them, what is the plan for each step, and how to improve the efficiency of surgery, this is very helpful for the safety of patients,” Fundamental VR Co-founder Chris Scattergood said.

The future of VR medical care will be a combination of the two: On the one hand, it is to teach medical students and professional physicians around the world how to perform a large number of routine surgeries. On the other hand, VR will also be used to allow higher-level physicians to understand some very rare surgical procedures.

Regardless of the direction, doctors can learn the necessary surgical skills in VR to maximize their own surgical skills before operating on real patients.

Original: http://

Gas Stove

Gas Stove,Single Burner Gas Stove,Tempered Glass Panel Gas Stove,Stainless Steel Gas Stove

Shandong Sangle Group Co.,Ltd. , https://www.sangle-group.com