Telestration: How Helena Mentis Applies Design Thinking to Surgery

Telestration: How Helena Mentis Applies Design Thinking to Surgery

Telestration: How Helena Mentis Applies Design Thinking to Surgery

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Read Next

Design Lab Cat Hicks Signalio

Cat Hicks Q&A: A Conversation about Google & her new start-up Signal IO

The Design Lab has long lasting impacts. Catherine Hicks has seen the Design Lab since its…

Design Lab Faculty

New Design Lab Faculty Working to Shape the Future of UC San Diego

As a global pioneer in design thinking, research, and invention, The Design Lab prides itself on recruiting the brightest and most innovative minds in the design field. Today, we would like to extend a warm welcome to brand new faculty members Elizabeth Eikey, Haijun Xia, and Edward Wang!

Elizabeth Eikey
From a first-generation undergraduate student at Penn State, then an inquisitive Best Buy employee and finally, to the Assistant Professor in the Department of Family Medicine and Public Health & The Design Lab at UCSD, Dr. Elizabeth Eikey has an illustrious career. Her research work at The Design Lab focuses on the intersection between technology, mental health, and equity, primarily studying the possible applications for technology in supporting mental health and well-being.

Haijun Xia
After receiving his PhD in Computer Science from University of Toronto, Xia made the move across countries to begin his time as a researcher at UC San Diego. ‘I wanted to work at The Design Lab and UC San Diego, because of the diversity of skill here,’ says the Professor, ‘We are all approaching the many challenging research questions from different angles, which is really important to develop comprehensive solutions.

Edward Wang
When Edward Wang was an undergraduate student at Harvey Mudd, he never expected himself to become a researcher, let alone becoming a professor. It was only after a Professor offered him the chance to help design a course she was planning about biosignal processing, that he began on this path. ‘As I was designing the class over summer, I had to read a bunch of papers,’ he says, ‘I couldn’t stop thinking about how cool all of it was. Especially when it branched out into computer science and how it could be involved in biosignal processes.’
Overcrowded Verganti Design

Don Norman: Overcrowded, by Roberto Verganti: In favor of criticism

I was just in Germany, in Herzogenaurach to be precise, at Adidas headquarters. (Hardly anyone knows where Herzogenaurach is — it’s a 20 minute taxi from Nuremberg.) I was at a conference organized by my old friend (and co-author) Roberto Verganti, from the business school at Politecnico di Milano. Years ago, he and I had a debate in Milan about the value of Human-Centered Design (HCD) and the way it is normally practiced. To the audience’s great surprise, we both agreed:

1. HCD is a powerful tool for improving existing products. That is, it is a powerful tool for incremental innovation.
2. HCD, by its very nature (hill-climbing plus a kind of design by committee), is a really bad tool for radical innovation.
Design Lab Michele Morris Greg Horowitt Scale

Smart Cities: Urban Innovation & Design Thinking

UC San Diego Design Lab Associate Director Michèle Morris recently joined venture capitalist, entrepreneur and…

Ucsd San Diego

Civic Design – DSGN 160: Open Enrollment

This studio course explores how to design products, services, and policies for complex socio-technical systems. The class will follow a human-centered design process that includes user research, concept generation, prototyping, pitching, and alliance building. Students will work in teams to design solutions to civic challenges affecting people in the San Diego region. This will be a good course for intermediate to advanced design students who want to build up their portfolio and to practice their skills with sketching, storyboarding, prototyping, and evaluating services for real-world problems.

As part of this class, teams will enter the Design for San Diego 2020 challenge (D4SD.org) and interact with other innovators, experts, and mentors throughout the city to address problems related to Mobility, Health, Environment, and Housing. Top teams will have an opportunity to present their work and win prizes at events in downtown San Diego!

Design Forward Panel Discussion: Focusing on 2017

UC San Diego’s Design Lab recently teamed up with Axure Software Solutions, Inc. to cosponsor…

Back To Top