A Virtual Reality Doctor Is Possible, But Is It Effective?

    by Amanda Enayati
    April 23, 2013
    On the right, virtual human agent "Ellie"; on the left, what is going on "under the hood" in terms of face, body and vocal sensing.

    Let’s all agree that there will be no more handwringing over whether robots are going to replace doctors in certain medical scenarios.

    Because the short answer is: yes.

    Connect the dots here and we may have an early iteration of a virtual reality doctor, a precursor of sorts to a real-life version of Star Trek's Emergency Medical Hologam Doctor."

    The virtual human health coach was science fiction a few short years ago. Now it’s fact, and it appears to work really well.


    Last fall I interviewed Skip Rizzo, associate director for medical virtual reality at University of Southern California’s Institute for Creative Technologies (ICT), for a talk I gave at Stanford’s Medicine X about the technologies being used to train athletes and soldiers to manage (and even harness) extreme stress.

    i-37a56f5349d765d90f63ad7d6b346cc0-Skip Rizzo smaller size.jpg

    Skip Rizzo develops gaming and virtual reality technologies to serve the clinical needs of soldiers.

    Rizzo and his colleagues were working on a number of fascinating projects, including SimCoach, a virtual human companion created to assist and guide active-duty military personnel and veterans suffering from stress, anxiety and post-traumatic stress disorder.


    If feeling stressed out, SimCoach users were able to select one of several avatars who, in turn, could provide more information about what the users may be experiencing. The coaches could also suggest local care facilities or perhaps walk the user through breathing exercises or stress reduction methods.

    SimCoach is not a “doc-in-the-box,” John Hart, ICT program manager, emphasized then. It will not make diagnoses, nor will it replace human interaction.

    Yeah, sure, I agreed, and yet it was hard to ignore SimCoach’s capacity to become an interactive virtual-reality source for all information on stress, anxiety and PTSD. And with the right technological advances, who knew …

    Meeting the Sensei … upon the Holodeck

    Seven months later, there has been a leap forward. Meet SimSensei, which is the SimCoach of yesteryear, souped up with cameras and Microsoft’s Kinect sensor to observe and analyze face, gesture and vocal parameters, all the better to understand your emotional state and react to it.

    i-cc3b6befcd8c1466ce40c14bca54ba5f-Ellie talking.jpeg

    Users can select one of several avatars to talk to when they are feeling stressed out.

    SimSensei is able to detect, with a high degree of accuracy, behaviors that are correlated to depression, says Stefan Scherer, a research scientist working on the project.

    When I reached out to Rizzo to discuss SimSensei, he told me they’ve moved the ball further still. For the past month he and his colleagues have been experimenting with the clinical and health applications of the Oculus Rift, which is being touted as the headset (with the greatest name ever) that may change the face of gaming to an immersive virtual reality. (Yes, we are talking about a kind of Holodeck.)

    Connect the dots here and we may have an early iteration of a virtual reality doctor, a precursor of sorts to a real-life version of Star Trek’s Emergency Medical Hologam Doctor. (Bonus points if developers can capture the Star Trek hologram doc’s vaguely sarcastic and highly humorous affect … On second thought, hold the sarcasm.)

    Enter Big Data

    “There is a Big Data movement that’s a critical part of this story,” Rizzo said. “We are at a point where we are capturing tons of information on a daily basis and we are increasingly able to analyze it in a way that may even surpass the human capacity to detect these signals.”

    i-a72bbf477b15fc2ab6d286c4a0afde3f-Big Data image version 5.jpg

    The fast-emerging voice of data has already begun to shift the ground beneath our feet in a number of other contexts, most notably during the elections last November, as well in our everyday lives.

    It is doing the same for the health interventions that Rizzo’s virtual reality Sensei are very good at performing: clinical assessment, treatment rehabilitation and resilience.

    Rizzo said while humans are gifted at capturing subtle nuances, scientists are beginning to match with virtual reality systems the human capacity to detect and interpret even the most minute social cues: facial expressions, body gestures, vocal parameters, pitch variability, frequency of speech and even pauses between words and sentences. In a way, he said, “that may even surpass the human capacity to detect these signals.”

    There is no doubt that humans have remarkable ability, but that ability can be fallible. We hold our biases and blind spots, and in some instances certain relationships may make us more vigilant or less sensitive. Computers’ ability to analyze countless signals opens a “psychological window into the soul,” Rizzo said.

    The notion of computers opening a window into my soul strikes me as surreal.

    Nonetheless, Rizzo continues: “You have to remember that computer systems are on 24/7. They are never fatigued, they don’t hold preconceived notions or biases, and they can continue to collect data, seek patterns and check for accuracy.”

    When might something like this be available to consumers, I asked.

    “Conservatively, three to five years.”

    The building blocks for empathy

    It is empathy that sets humans and androids apart in Phillip Dick’s classic science fiction novel, “Do Androids Dream of Electric Sheep?

    And in the real world, too, it is empathy, that uniquely human gift, which would seem to set the flesh-and-blood doctor apart from her virtual counterpart and, according to some, doom the latter to failure.

    i-9bfb563d9cf2ab1a18466fdb56aec706-Bill Ford image.png

    SimCoach Avatar Retired Sergeant Major Bill Ford

    “We are not building digital therapists, nor are we trying to replace clinicians,” Rizzo said, echoing Hart’s earlier caution, “but we are creating the building blocks for a system that can mimic empathy.”

    I recall a story I heard on NPR about a man who suffers from Asperger Syndrome and sets out to teach himself how to mimic “normal” behavior. In an attempt to become more responsive to social cues, he creates a journal of “best practices” about a variety of everyday circumstances, including how to be a better husband to his wife and a better father to his two young children.

    That man was teaching himself how to respond to social cues in order to properly express feelings that he actually felt. A computer, on the other hand, may be programmed with “best practices” for showing empathy, but it could never actually feel them. The idea of synthetic empathy bothers me, in the same way that fake love and fake flowers and fake food might.

    Will the “real” healer please stand up

    But maybe we are asking the wrong questions here. Perhaps in the real world, a virtual doctor’s inability to feel empathy will have little or no bearing on the tasks it will be performing.

    So much of the resistance against the disruption taking place in medicine right now smacks of fear of uncertainty and change. However, as the poet Khalil Gibran observed, “life goes not backward nor tarries with yesterday.”

    Is it possible for a virtual reality physician to be programmed with best practices of how to handle a wide variety of medical scenarios?

    From all indications, yes.

    And could there even be circumstances where the virtual reality doc may trump its flesh-and-blood counterpart?

    Sure, particularly when it comes to certain repetitive diagnostic and analytical functions.

    But could a virtual doc replace a real one?

    As a physician observed in his recent blog post, “Will robots reduce the need for doctors?“: “Computers cannot provide the healing touch. Computers cannot comfort … The human relationship really matters to most patients.”

    In the end, it is not an either/or proposition. It never has been. And particularly, if you define “doctor” as a healer with scientific knowledge, a compassionate heart and a keen intuition, then he or she will never be replaceable. Not even on “Star Trek.”

    Amanda Enayati is the technology and stress correspondent for PBS MediaShift as well as CNN Health’s stress columnist.

    Tagged: big data clinical virtual reality depression oculus rift ptsd simsensei stress virtual reality doctors

    Comments are closed.

  • Who We Are

    MediaShift is the premier destination for insight and analysis at the intersection of media and technology. The MediaShift network includes MediaShift, EducationShift, MetricShift and Idea Lab, as well as workshops and weekend hackathons, email newsletters, a weekly podcast and a series of DigitalEd online trainings.

    About MediaShift »
    Contact us »
    Sponsor MediaShift »
    MediaShift Newsletters »

    Follow us on Social Media