Telestration: How Helena Mentis Applies Design Thinking to Surgery

Telestration: How Helena Mentis Applies Design Thinking to Surgery

Telestration: How Helena Mentis Applies Design Thinking to Surgery

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Read Next

Tricia Ngoon

Tricia Ngoon, UCSD & Design Lab PhD Graduate, Discusses “Adaptive Conceptual Guidance”

Currently, in the spotlight of Tricia Ngoon’s research and involvement with The Design Lab is her recently accepted paper, Shöwn: Adaptive Conceptual Guidance Aids Example Use in Creative Tasks, which will appear in the Designing Interactive Systems virtual conference this summer, 2021. Her research hypothesizes that providing “adaptive conceptual guidance” will improve a person’s implementation of examples within creative work, as opposed to providing a static example. Using the domain of web comics, “[researchers in the study] present concepts to people alongside examples as they work.” Ngoon adds that “It’s essentially a step towards coaching. For example, if [a person is] working on a comic you might present a concept to consider the framing or kind of the composition of the panel and then [show] examples of different types of framing and composition.” Ultimately, her research concluded that “these adaptive suggestions as a person is working in context really help with making a clear and more unique story. It kind of changes the way they look at their ideas because they are more likely to explore different [ones].” 
Don Norman On User-friendly Design

I wrote the book on user-friendly design. What I see today horrifies me

The world is designed against the elderly, writes Don Norman, 83-year-old author of the industry bible Design of Everyday Things and a former Apple VP.

More people than ever are living long, healthy lives. According to the U.S. Department of Health and Human Services, the average life expectancy is 78.6 years for men and 81.1 for women. More relevant, however, is that as people grow older, their total life expectancy increases. So for those who are now 65, the average life expectancy is 83 for men and over 85 for women. And because I’m 83, I’m expected to live past 90 (but I’m aiming a lot higher than that). And these are averages, which means that perhaps half of us will live even longer.

Opinion: The World Design Capital is more than an award. It’s a chance to solve problems.

San Diego Union Tribune Op-Ed by Tad Parzen and Eddie Matthews*
*Eddie Matthews is also a Designer-in-Residence with the UCSD Design Lab

Design and discovery are in our San Diego-Tijuana region’s DNA. We are risk-takers, a nimble community that has long fostered life-changing design and innovation to improve the world, save lives and sustain critical resources. San Diego-Tijuana has long been the epicenter of a binational design revolution.

In this inclusive spirit, a cross-border community of designers, activists and community leaders have joined together to submit the first binational bid to the World Design Organization to name San Diego-Tijuana the World Design Capital in 2024. This designation recognizes cities for their effective use of design to drive economic, social, cultural and environmental development, and showcases best practices in sustainable, human-centered policy and innovation.

Every member of this region has something to contribute and the World Design Capital will be the centralized place for these binational design contributions, making the border immaterial by making design visible.
San Diego And Tijuana Selected As 2024 World Design Capital

San Diego And Tijuana Selected As 2024 World Design Capital

US-Mexico Border Cities Win Historic Designation to Become First Binational Design Capital

SAN DIEGO-TIJUANA - Today the World Design Organization (WDO) announced the San Diego-Tijuana region has been selected as the World Design Capital (WDC) for 2024.

The selection of the San Diego-Tijuana joint bid makes it the first binational World Design Capital in WDO history. While the designation is for both cities as a united region, San Diego is now the first U.S. city ever to receive the WDC designation. Tijuana is the second city in Mexico to hold the title, following Mexico City in 2018.

"We did it!" said Don Norman, founder of UC San Diego’s Design Lab (now retired) and co-founder and Board advisor to the Design Forward Alliance (DFA). "Designers, city officials, and organizations in both the Tijuana and San Diego regions collaborated to make our binational community the World Design Capital for 2024. It shows the power of design as a way of thinking, to address important societal issues, and as a source of innovation for companies, organizations, and educational communities at all levels. We have built a permanent coalition of our communities to address civic and climate challenges, to grow our industrial sectors, and to support a strong culture of cross-border design."
Don Norman Emphatic Design

Why I Don’t Believe in Empathic Design

Human-centered design pioneer Don Norman, who coined the term ‘user experience,’ explains why he’s not convinced by the current obsession with empathy and what we should do instead.

I approve of the spirit behind the introduction of empathy into design, but I believe the concept is impossible, and even if possible, wrong. The reason we often talk about empathy in design is that we really need to understand the people that we’re working for. The idea is that, essentially, you’re in a person’s head and understand how they feel and what they think.
Design Lab Don Norman Healthcare Designforward

San Diego is Getting Serious About Healthcare Design

In June 2017, San Diego hosted two of the largest annual healthcare conventions - the…

Back To Top