“Language is a lens into the mind,” says U of T computer scientist Frank Rudzicz. “How you’re feeling, what you’re thinking, what your cognitive performance is – it’s all revealed through your language.”
Language, in the form of a speech disorder, is also a lens into the minds of patients suffering from cerebral palsy, Parkinson’s disease and multiple sclerosis. It is one of the telltale symptoms of Alzheimer’s disease and one of the first signs of cognitive decline as a person ages.
Almost 10 per cent of the North American population has some form of speech disorder, including 7.5 million individuals with disorders caused by cerebral palsy, Parkinson’s or multiple sclerosis. By 2030, fully a quarter of Canadians will be over 65 years of age – up from 16 percent in 2014 – and with this aging population, an increase is expected in speech and language disorders associated with age-related cognitive decline.
The numbers put into stark contrast the growing need for early detection, accurate diagnosis and improved outcomes in care, and the potential for language to aid in all those facets. And now, there is a new tool for researchers, health care professionals and educators doing research into or assessment of such speech disorders.
Talk2Me is a web portal that gathers linguistic data through an array of cognitive tasks performed by participants. Researchers can access the gathered data for their research and they can also use the portal to gather their own data from select participants using particular tasks.
Talk2Me will help enable a community of people to solve problems related to neuro-degenerative issues, cognitive issues and psychiatry.
“Talk2Me will help enable a community of people to solve problems related to neuro-degenerative issues, cognitive issues and psychiatry,” Rudzicz says. “It’s a common, open platform to help solve these problems.”
In a diagnostic setting, the portal, which also runs on a tablet, replaces the typical assessment scenario conducted with pen and paper between a physician and their patient – a scenario that can be imprecise and vulnerable to bias.
The tool was developed by a team that includes Rudzicz, a professor in the Department of Computer Science in the Faculty of Arts & Science at U of T. Rudzicz is also affiliated with the Vector Institute for Artificial Intelligence, the International Centre for Surgical Safety Technologies, Li Ka Shing Knowledge Institute St. Michael’s Hospital, and Winterlight Labs.
His Talk2Me collaborators include Daniyal Liaqat, a PhD candidate in the Department of Computer Science, as well as researchers from Carleton University, Saint Michael’s Hospital, the National Research Council and Winterlight Labs. They described Talk2Me in a paper published March 27th by PLOS One.
“What was too big, the trophy or the suitcase?”
Talk2Me collects data using tasks similar to those used in standard assessments of cognition, with participants inputting their responses by typing or speaking.
For example, in the picture description task, participants describe images like the “cookie theft” illustration that portrays a woman and two children in a kitchen. The woman is washing dishes while two children take cookies from a cookie jar. In this task, participants typically respond with varying degrees of detail and inference. Some identify the woman as the mother even though the relationship is not explicitly portrayed; similarly, the motivation of the children is unclear but some say they are stealing cookies behind their mother’s back.
In the Winograd schema task, participants are given a statement like: “The trophy could not fit into the suitcase because it was too big.” They are then asked: “What was too big, the trophy or the suitcase?” Responding that the suitcase is too big could be a sign that a person’s executive function – defined by our set of mental skills – is impaired. If a person’s ability to answer properly changed over time, it could be an indication of the onset of age-related dementia.
Other tasks require participants to type as many words as possible that fit in a given category; for example, “fruit” and “apple.” In another, they are asked to retell a short story they have just read. And in the word-colour Stroop task, participants are shown the name of a colour spelled out in coloured letters – for example, the word “green” spelled out in a red typeface. Participants are asked – not to read the word – but say the colour of the letters; in this case, “red.”
Different tasks involve different mental processes and the responses contain different “features” or “measurable units of language.” Talk2Me’s natural language processing analyzes text and audio for these features, which include the number of words used to describe something; the number of syllables in words; grammatical complexity; the frequency of speech fillers like “uh” and “um”; pitch; pauses; loudness; and more.
Improving assessments, outcomes and lives
Rudzicz’s broad focus is to apply natural language processing and machine learning to health care, and Talk2Me is just one way in which he and his collaborators are studying cognitive health through the lens of language.
“It’s an exciting time now where artificial intelligence can make a real impact in health care,” he says. “And my colleagues and I want to have an impact beyond publishing papers and academic output. We want to change lives for the better and improve outcomes.”
In 2015, Rudzicz co-founded Winterlight Labs along with computer scientists Katie Fraser, Liam Kaufman and Maria Yancheva. Winterlight is a U of T start-up designing tools to track, screen for and predict the onset of dementia and psychiatric illness. Their first product was a tool that runs on a tablet or computer that, like Talk2Me, gathers input from a patient and analyzes the data to help in diagnosing and predicting Alzheimer’s disease.
With Talk2Me, and their work at Winterlight and other institutes, Rudzicz and his colleagues exploit the lens of language, and continue to sharpen its focus to see more clearly into the human mind.