Augmented Reality Takes Simulation Learning to 3-D Level

USF College of Nursing educators took a test drive of the latest high-tech teaching tool in experiential learning last week — a cutting-edge ultrasound simulator that projects 3-D holographic images.

Manufacturers of the CAE VimedixAR with Microsoft HoloLens visited the nursing school on May 17 for a demonstration showcasing the innovative technology featuring augmented reality.

The augmented reality technology allows users wearing the HoloLens headgear to visualize complex organs. Holographic layers of the circulatory, respiratory, and skeletal structures leap to life while users walk around the manikin.

Experiential learners can peer inside the 3-D image of the human heart and see the valves and arteries. The innovative technology allows users to pinch the hologram and enlarge, minimize, or rotate it to see how its structures are interrelated.

“This is really the future of education and training,” said Teresa Gore, PhD, DNP, FNP-BC, NP-C, CHSE-A, Assistant Dean of Experiential Learning and Simulation at the College of Nursing. “We want to do this so we can look at the best way to educate our nurses and provide the best care for their patients.”

The ultimate goal with this technology is to improve patient safety and outcomes.

Dr. Gore said the college plans to purchase the ultrasound simulator with the Microsoft HoloLens technology and incorporate the teaching tool into classrooms beginning in the Fall of 2018.

The simulator provides up to 200 different physiological cases to help students diagnose medical conditions using an ultrasound scanner. The system will also help instructors teach undergraduate and graduate nursing students about anatomy and physiology.

Photo courtesy of CAE Healthcare

Gore said the HoloLens component, where the traditional images from an ultrasound scan are projected as 3-D holograms, takes experiential learning to the next level.

“As you move the ultrasound probe through the manikin, you see the body as it slices,” Gore said. “It is seeing it in motion and in action. It is that augmentation that you see that makes you learn better.”

As ultrasound imaging becomes more widely used as a diagnostic tool in many medical specialties, the ultrasound simulator can be a great training tool, said Nicole Sherk, a regional sales manager with the Sarasota-based simulator manufacturers CAEHealthcare.

“They can become more comfortable with learning how to scan, learning how to capture views, learning how to recognize any sort of abnormalities or patient complications on a simulator versus doing it on a real patient during their clinical rotations,” Sherk said.

The HoloLens technology allows users to peer inside the patient and see a 3-D representation of the vital organs — something that they won’t be able to see on a traditional ultrasound image.

“They now have the ability to put the goggles on and look at it in an augmented reality,” she said. “They can move the anatomy around and actually stick their head inside of a heart. They can rotate their head underneath the heart and be able to see how that heart is functioning from a different perspective.”

Story by Elizabeth L. Brown, USF College of Nursing