skip to Main Content

Telestration: How Helena Mentis Applies Design Thinking to Surgery

Telestration: How Helena Mentis Applies Design Thinking to Surgery

Telestration: How Helena Mentis Applies Design Thinking to Surgery

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Read Next

Anna McCowan Melanie McComsey Ucsd Design Lab

UK Markey Center and UCSD Seek to Improve Cancer Care with LAUNCH

According to the Centers for Disease Control and Prevention (CDC), Americans living in rural areas are more likely to die of cancer than their counterparts in urban settings, which sets them apart from the many communities nationwide that have experienced a 20 percent decrease in cancer mortality over the past two decades. In Appalachia, the cancer picture is bleaker than in other rural parts of the country. Between 1969 and 2011, cancer incidence declined in every region of the country except rural Appalachia, and mortality rates soared.

This week (Monday, June 17th, 2019) an Innovation Studio workshop was held at the PRTC center announcing a program called LAUNCH.
Design Lab Michele Morris Forward

Why Does Design Matter: A Q&A With Design Forward Founder Michèle Morris

Michèle Morris currently serves as the Associate Director of the Design Lab at UC San Diego. She…

Olga McConnell

Olga McConnell, Project Specialist and Executive Assistant to the Director of The Design Lab

As the Executive Assistant to the Director of The Design Lab, a project manager for the Lab’s special projects and annual events, and a lifelong learner who holds a M.A. in English Linguistics and Translation, and a M.B.A. in Business Administration and Management, Olga McConnell’s zest for knowledge is palpable. She is currently on track to complete a Project Management Certification at UC San Diego Extension at the end of 2021, and she is planning on obtaining her Project Management Professional (PMP) Certification after that. “I’m kind of addicted to getting degrees,” jokes McConnell. “I even thought the other day, maybe I’ll go to law school. And then I was like, no, enough, enough.” 

For nearly five years, McConnell was Executive Assistant to Don Norman, the Founding Director Emeritus of The Design Lab. She is now the Executive Assistant to the new Director of The Design Lab, Mai Thi Nguyen. It is Nguyen’s vision of human-technology-community interactions, along with her JEDI (justice, diversity, equity and inclusion) approach that has McConnell excited about this new chapter in the Lab’s legacy, saying, “I see how great she is as an efficient leader, so I’m really looking forward to working with her, supporting her administratively, as well as taking charge of certain projects that she has in mind.”

Nazima Ahmad is Putting People Over Profits by Connecting Art and Design at the Design Lab

If the COVID-19 pandemic has taught us anything, it’s that our communities are more important than ever. The pandemic has changed the ways in which we experience and perceive community—sometimes even causing us to feel that it has vanished. The past two years have been an uncertain time for those involved in the arts, with many creative professionals being impacted by dwindling audiences and interest. Nazima Ahmad, a Designer-in-Residence at The Design Lab, noticed the crumbling connection between artists and consumers and sought to find a way to mend it. Working with fellow designer Michelle Hoogenhout, the two were able to come up with City Canvas, a concept developed in collaboration with the Arts and Commission Department of the City of San Diego that won the 2020 SCALE San Diego Urban Innovation Challenge that works to make connecting with local artists easier for San Diego residents.

“Working with the city on that project was all-around trying to figure out how to promote the creative economy of San Diego,” says Ahmad of the goal of the project. Not only was the project a great success at Design Week, but it is also what led Ahmad to The Design Lab.
Albert Lin National Geographic

Design Lab member Albert Lin hosted 3 National Geographic series, using technology to uncover lost cities, treasures, and secrets

Full length episodes available for streaming on the Disney+ app.

Lost Cities With Albert Lin: “Lost Cities brings adventure, science, and archaeology together through our host Albert Lin. Our ambitious approach applies 3D scanning to some of the most extraordinary sites of antiquity."

Lost Treasures Of The Maya: “National Geographic Explorer Albert Lin ventures into the Guatemalan jungle to explore how a new high-tech treasure map is revealing tens of thousands of ancient ruins.”

Buried Secrets of The Bible With Albert Lin: “Albert Lin seeks out the truth behind two great stories of the Bible. To solve these mysteries, Albert will use satellites and space-age technology to look beneath the earth’s surface to reveal secrets that have been buried for thousands of years.”

UCSD Design Lab & the National Cancer Institute organize workshop on Human Systems Integration

On October 20 and 21, the Design Lab jointly organized a workshop with the National…

Back To Top