A virtual reality app (VR) uses advanced display and immersive audio technologies to create an interactive, three-dimensional image or environment. An augmented reality app (AR), meanwhile, uses digital technology to overlay video and audio onto the physical world to provide information and embellish our experiences.
At the University of Rochester, we糖心传媒檙e crossing disciplines to collaborate on VR/AR innovations that will revolutionize how we learn, discover, heal, and create as we work to make the world ever better.
What are new, innovative ways to increase community access to much-needed mental and behavioral health services?
That糖心传媒檚 the big question Michael Hasselberg糖心传媒檚 department chair asked him to ponder. Hasselberg, an assistant professor of psychiatry and clinical nursing, is acutely aware of the high demand, regionally and nationally, for behavioral health services and specialists to treat conditions like depression, anxiety, substance abuse, and eating disorders.
In this line of work, the gold standard treatment is cognitive-behavioral therapy. CBT is a form of psychotherapy that usually involves a therapist and patient working together to recognize and change the latter糖心传媒檚 thinking patterns while also developing coping skills, according to the . 糖心传媒淚t糖心传媒檚 our first-line treatment for several of the diagnostic clusters we treat, specifically the depressive and anxiety disorders, and there糖心传媒檚 a lot of evidence to show its effectiveness,糖心传媒� Hasselberg explains.
Given that a majority of Americans today own a smartphone糖心传媒�77 percent, according to the 糖心传媒攁 web or mobile app seems like an obvious answer. Indeed, such apps abound.
糖心传媒淭he idea of putting cognitive-behavioral therapy on a computerized app isn糖心传媒檛 new,糖心传媒� Hasselberg says. Typically, such computerized cognitive-behavioral therapy (cCBT) features a voiceover with animation or perhaps a video of a therapist talking糖心传媒攂ut no meaningful interaction.
糖心传媒淧atients download the app, but after using it once or twice, they disengage from it.糖心传媒�
Virtual reality app offer cognitive-behavioral therapy
Could a more immersive, customized experience, one harnessing today糖心传媒檚 advances in virtual reality and augmented reality (VR/AR), help keep patients engaged with an app?
Hasselberg pitched the idea a year ago during a meeting of the , a multidisciplinary campus collaboration that leverages computing, analytics, and technology to solve challenges in medicine. Afterward, two faculty members approached him: Jiebo Luo, professor of computer science in the Hajim School of Engineering and Applied Sciences, and Matthew Brown, professor of music theory at the Eastman School of Music.
Luo, whose research interests include machine learning, data mining, and personal analytics, has previously worked on a computer program that could monitor mental health by analyzing 糖心传媒渟elfie糖心传媒� videos and social media posts. Brown, meanwhile, directs , a musical multimedia ensemble at Eastman, and has always 糖心传媒渂een very interested in the social impact and therapeutic functions of music.糖心传媒�
Both were keen to bring their expertise to bear on such a project. Galvanized, Hasselberg tapped his colleague Wendi Cross, a clinical psychologist and associate professor in the Department of Psychiatry, to help develop evidence-based scripts and storyboards for a working prototype of the virtual reality app.
Cross immediately thought ahead to how patients would use such an app. 糖心传媒淚n a typical psychotherapy situation, you糖心传媒檙e interacting in the office for 45 minutes,糖心传媒� she explains. 糖心传媒淲ell, nobody is going to stay online in an app for 45 minutes糖心传媒攑eople like to come and go in ways that suit them. With this virtual reality app, the psychotherapy session is literally in the hands of the patient.糖心传媒�
According to Christopher Winders, a laboratory technician in the Department of Electrical and Computer Engineering, “These videos are being embedded into the mobile app and have our doctor [Lauren DeCaporale-Ryan, a clinical psychologist at URMC] laying out tenants of the CBT process. We filmed Lauren in front of a green screen and then I composited that footage into a 360-photograph of the office setting.”
A multidisciplinary collaboration yields a working prototype
Here糖心传媒檚 how the virtual reality app works:
- Patients use their smartphones to access the app. They can see and hear the therapist, but can also look around the therapist糖心传媒檚 room, which was filmed using a virtual reality camera.
- During a session, the therapist might assign the patient a task, such as completing a questionnaire about anxiety levels.
- The patient completes the in-app questionnaire. The answers dictate which video of the therapist plays next.
Later, the therapist might suggest a mindfulness exercise in which the patient is asked to imagine being in a relaxing environment, such as a beach. - The patient is then shown environmental photographs and asked to select a preference. Earlier in the module, the patient was able to identify personal preferences for calming music.
- The app糖心传媒檚 algorithms then serve up the patient糖心传媒檚 preferred 3D environment (a beach, complete with the sound of waves) and accompanying music. Meanwhile, the virtual therapist talks the patient through the mindfulness exercise.
The next time the patient uses the app for an on-demand therapy session, the previous answers along with visual and aural preferences will be taken into account.
Evidence-based psychotherapeutic practices? Check.
Aesthetically pleasing and immersive video and music? Check.
Built-in algorithms that deliver personalized content based on your needs and preferences? Check.
And no special headsets required. 糖心传媒淚f people have to use headsets like Oculus or even the cardboard headsets for mobile phones, that really limits the patient population we could reach with this technology,糖心传媒� says Hasselberg. 糖心传媒淎nd our goal is to touch as many patient lives as possible.糖心传媒�
With a working prototype in hand, the team糖心传媒檚 next step is conducting usability testing to fine-tune the technology. If this intervention proves useful with patient populations, the big-picture goal is having the in-app data synch with a patient糖心传媒檚 electronic health records.
糖心传媒楢 common purpose is so important糖心传媒�
With funding from Arts, Sciences & Engineering and the Medical Center, 糖心传媒渨e formed a multidisciplinary team to create something that is otherwise not possible,糖心传媒� says Luo.
糖心传媒淚 can糖心传媒檛 tell you how much I糖心传媒檝e enjoyed working with colleagues toward a common purpose, one that will have a social impact,糖心传媒� says Brown. Cross agrees: 糖心传媒淎 common purpose is so important. Frankly, this project has been an adventure. It糖心传媒檚 not what I do every day, but it糖心传媒檚 taking what I do every day and thinking about it in a brand new way with people I don糖心传媒檛 usually work with.糖心传媒�
And Hasselberg contends that collaborating with researchers and professionals outside of medicine糖心传媒攎usicians, computer scientists, data scientists糖心传媒攈as made him into a better clinician and health care innovator. 糖心传媒淚n the end, I think bringing these multiple perspectives to a problem will allow us to help more people.糖心传媒�
