Telestration: How Helena Mentis Applies Design Thinking to Surgery

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Upcoming Events

D4SD Hackathon Kick-off

Friday, September 22, 2017 at 5:30 p.m. Don’t miss out! The San Diego Union-Tribune just issued our call to action: “Public invited to all-night hackathon to solve San Diego’s commuting nightmare.” It’s all about our Design for San Diego (D4SD) challenges that kickoff this Friday at 5:30 p.m. at Downtown Works. Join us for this incredible opportunity to collaborate and compete with other […]

Design Forward Summit 2017

October 25-27, 2017 Human-centered design has the power and potential to drive economy, industry, civic infrastructure, and quality of life. Design Forward 2017 is the lens to see it happen. Entrepreneurs, technologists, product managers, designers, business and civic leaders are all welcome. Visit the Design Forward website for more information and to purchase tickets.

Studio Session with IBM’s Talent and Design Teams

Friday, June 9, 2017 at 12:00 PM – 1:30 PM UCSD The Design Lab, Atkinson Hall, 1601 UC San Diego students are invited to join us in the Design Lab for lunch and listen to the IBM Teams discuss what differentiates IBM in the tech industry, and how you can get ahead! Come with questions […]

Join Us

Are you a Design practitioner, student, or industry professional interested in getting involved with the Design Lab? Contact us, we want to hear from you.

Contact Us