Machine learning lets Rochester researchers accurately identify signs of the neurological disease by analyzing facial muscles.
What are the signs and symptoms of Parkinson糖心传媒檚 disease?
Although individuals may experience symptoms differently, the four common signs of Parkinson糖心传媒檚 disease are:
- Muscle rigidity or stiffness when the arm, leg, or neck is moved back and forth.
- 罢谤别尘辞谤蝉糖心传媒�involuntary movement from contracting muscles糖心传媒攅specially when at rest.
- Slowness in initiating movement.
- Poor posture and balance that may cause falls or problems with walking.
Get more information about Parkinson糖心传媒檚 disease from .
Every day, millions of people take selfies with their smartphones or webcams to share online. And they almost invariably smile when they do so.
To and his collaborators at the 糖心传媒, those pictures are worth far more than the proverbial 糖心传媒渢housand words.糖心传媒� Computer vision software糖心传媒攂ased on algorithms that the computer scientist and his lab have developed糖心传媒攃an analyze the brief videos, including the short clips created while taking selfies, detecting subtle movements of facial muscles that are invisible to the naked eye.
The software can then predict with remarkable accuracy whether a person who takes a selfie is likely to develop Parkinson糖心传媒檚 disease糖心传媒攁s reliably as expensive, wearable digital biomarkers that monitor motor symptoms. The researchers糖心传媒� technology is described in .*
糖心传媒淧arkinson糖心传媒檚 is the fastest growing neurological disorder,糖心传媒� says Hoque, an associate professor of . 糖心传媒淲hat if, with people糖心传媒檚 permission, we could analyze those selfies and give them a referral in case they are showing early signs?糖心传媒�
Though ethical and technological considerations still need to be addressed, the has agreed to fund this novel research through a $500,000 grant, effective November of 2021.
糖心传媒淭he foundation wants us to validate the feedback that we would give people if they did, indeed, show early signs of Parkinson糖心传媒檚糖心传媒攅specially if they are performing the test at home,糖心传媒� Hoque says. 糖心传媒淭he challenge is not only validating the accuracy of our algorithms but also translating the raw machine-generated output in a language that is humane, assuring, understandable, and empowering to the patients.糖心传媒�
Software analyzes facial expressions, hand movements
Smiles are not the only behaviors that Hoque and his lab can analyze for early symptoms of Parkinson糖心传媒檚 disease or related disorders.
In collaboration with 糖心传媒攁 leading expert in Parkinson糖心传媒檚 disease and the David M. Levy Professor of Neurology at Rochester糖心传媒攁nd the University糖心传媒檚 , the researchers have developed a five-pronged test that neurologists could administer to patients sitting in front of their computer webcams hundreds of miles away.
This could be transformative for patients who are quarantined, immobile, or living in underdeveloped areas where access to a neurologist is limited, Hoque says.
In addition to making the biggest smile, and alternating it with a neutral expression three times, patients taking the test are also asked to:
- Read aloud a complex written sentence
- Touch their index finger to their thumb 10 times as quickly as possible
- Make the most disgusted look possible, alternating with a neutral expression, three times
- Raise their eyebrows as high as possible, then lower them as far as they can, three times slowly
Using machine learning algorithms, the computer program shows糖心传媒攚ithin minutes糖心传媒攁 percentage likelihood from each of the tests whether the patient is showing symptoms of Parkinson糖心传媒檚 disease or related disorders.
What exactly does the program look for? When patients are making a smile, the software can detect whether they show less control over their facial muscles while doing so, a symptom of Parkinson糖心传媒檚 that clinicians refer to as 糖心传媒渕odularity.糖心传媒�
糖心传媒淥ne thing about Parkinson糖心传媒檚 is that you don糖心传媒檛 show all the symptoms all the time, and not every symptom is shown in every part of your body,糖心传媒� says Rafayet Ali 糖心传媒�20, lead author of the paper. 糖心传媒淔or example, you may not have hand tremors, but you may show a significant level of deviation in your smile.糖心传媒�
Hence the importance of testing other expressions and movements, according to Ali, a former postdoctoral associate in Hoque糖心传媒檚 lab who now is an associate data scientist at Sysco.
From pen-and-paper evaluations to 糖心传媒榦bjective, digital assessments糖心传媒�
Both Hoque and Ali have personal stakes in helping people with Parkinson糖心传媒檚. Their mothers both have suffered from the disorder. Hoque糖心传媒檚 late mother in Bangladesh, for example, was put on leovodopa, a leading medication for the disorder, after finally finding one of the country糖心传媒檚 few neurologists. The tremors went away. 糖心传媒淲e were so happy,糖心传媒� Hoque says. Unfortunately, it was difficult to make follow-up appointments, and the tremors eventually returned.
That prompted Hoque to email Dorsey, 糖心传媒渏ust to casually chat.糖心传媒� When they finally met in 2016, Hoque recalls, Dorsey 糖心传媒渢ook a big document and just threw it on the table.糖心传媒� The pamphlet contained forms physicians need to fill out as part of the Movement Disorders Society糖心传媒揢nified Parkinson糖心传媒檚 Disease Rating Scale (MDS-UPDRS).
糖心传媒淭hat糖心传媒檚 the gold standard for measuring Parkinson糖心传媒檚,糖心传媒� Dorsey told him. 糖心传媒淓verything we do is pen and paper. Any automation, any data analytics that you can bring into this would be a contribution. And he immediately helped us see how we could do that,糖心传媒� Hoque says.
糖心传媒淥bjective, digital assessments of Parkinson糖心传媒檚 disease can help us diagnose people with the condition and evaluate new therapies for the condition faster,糖心传媒� says Dorsey, an author of (2020).
Progress toward FDA approval
It will be a while yet, however, before Hoque and his researchers can start seeking permission to analyze people糖心传媒檚 selfies, or even before neurologists can deploy the five-pronged test that the researchers have developed.
糖心传媒淎n algorithm will never be 100 percent accurate,糖心传媒� Hoque says. 糖心传媒淲hat if it makes a mistake? We want to be very careful and follow guidance from the FDA if we want anybody from any part of the world to try this and get an assessment.糖心传媒�
Moreover, there is a whole family of movement disorders that are closely related to Parkinson糖心传媒檚 disease, including ataxia, Huntington糖心传媒檚 disease, progressive supranuclear palsy, and multiple dystrophy.
糖心传媒淭hey all share similar symptoms of tremor, but the tremors are very different in nature,糖心传媒� Hoque says. 糖心传媒淗owever, even expert neurologists find it very, very difficult to distinguish among them.糖心传媒�
The researchers have made great progress in detecting Parkinson糖心传媒檚 disease by automatically analyzing expressions, voice and motor movements. Yet further work is needed to develop algorithms to differentiate how these involuntary tremors differ across other movement disorders, including Ataxia and Huntington糖心传媒檚.
糖心传媒淲e can糖心传媒檛 tell that just yet,糖心传媒� Hoque says. 糖心传媒淏ut we are in a pursuit of differentiating those tremors using AI to prevent the potential harm of misdiagnosis while maximizing benefit.糖心传媒�
And speaking of timely Parkinson糖心传媒檚 disease diagnosis . . .
Turns out it糖心传媒檚 not just selfies and videos that can help with diagnosing Parkinson糖心传媒檚 disease.
More and more, people are using speech-activated smart devices, such as Alexa, Apple Watch, and Google Voice Assistant, to accomplish everyday tasks. Could these devices analyze our speech and voices to alert us if we show early warning signs of Parkinson糖心传媒檚 disease?
Recent work by Rochester researchers suggests it糖心传媒檚 entirely possible. Wasifur Rahman, Sangwu Lee, Md. Saiful Islam, and other students in Hoque糖心传媒檚 lab published findings in the Journal of Medical Internet Research that show how an online tool can be used to help screen almost anyone anywhere for Parkinson糖心传媒檚 disease remotely using video- or audio-enabled speech tasks.
Taken together, the Rochester researchers糖心传媒� efforts are contributing to a future in which equity and access to neurological care is as ubiquitous as owning a smart phone or other internet-enabled device.
*Editor糖心传媒檚 note: As of February 2023, the authors have retracted the article published in聽npj Digital Medicine聽that established聽the relationship between smiles and the early onset of Parkinson糖心传媒檚 disease because the classification described in the paper was performed on an inaccurate use of a data pre-processing tool. Please see the authors糖心传媒櫬� for more information. In June 2025, a was published in NEJM AI聽demonstrating that smiling videos can effectively differentiate between individuals with and without Parkinson糖心传媒檚 disease.
