Vincent Van Gogh's paintings might not make it obvious that he was an artist troubled with depression and mania. But a computer algorithm might be able to figure that out. Computer programs are getting pretty good at discovering health information by studying heaps of social media data.
A computer script analyzed galleries of photos posted to Instagram and accurately predicted if the users had depression, according to a study posted this month to the public online repository arXiv.com.
The researchers asked 166 Instagram users for permission to analyze their posts and also asked whether or not they had a diagnosis of clinical depression from a mental health professional. First, the researchers had their algorithm just sort 70 percent of the information to find out what photo features are common among people with depression and people without depression.
The program looked through four distinct categories of information: color and brightness, number of faces in the photo, use of Instagram filters, and metadata including number of comments or likes. "In general, each one of these things has some relationship to what people have already found in studying depression," says Andrew Reece, the lead author and a psychology and computational science graduate student at Harvard University.
For example, there's some research that suggests people with depression prefer darker colors and more grays or blues. "There's evidence people who are depressed tend to interact in smaller social settings. If there's a ton of faces in the photos, then it probably means you're interacting in large social settings," Reece says. The same might be true for the number of likes or comments someone gets for their posts.
The program was able to find associations in the Instagram data that backed these theories up. Photos from people with depression tend to be bluer, darker and grayer. "[People with depression] tend to have more comments on their posts, but fewer likes.
Depressed people were less likely to use any filters at all, but when they did use filters they went for Inkwell, which makes everything black and white," Reece says. "And depressed people have fewer faces in their photos, but they tend to post more photos with faces."
Then Reece and his coauthor removed the depression diagnosis information from their remaining untouched Instagram photos. They gave those to the computer program and had the machine figure out which of those users had depression and which of them didn't. "We were right about people being healthy 84 percent of the time. About a third of the people who were truly depressed, the machine found," Reece says.
When the machine gave a depression marker, it was right about 54 percent of the time, compared to unassisted primary physicians who correctly make a depression diagnosis about 42 percent of the time. "It's not an A+, but it's a 25 percent improvement over those human rates," Reece says.
This study hasn't been peer reviewed, but it looks like the algorithm is doing a a fairly good job. Reece says it doesn't mean that computer programs are going to replace primary doctors when it comes to diagnosing depression. Primary doctors look at the general population, rather than just Instagram users, and don't separate people into binary categories like depression or no depression like the program does. "You can't even really compare the two in a formal sense," Reece says.
And the algorithm can't truly diagnose someone with depression or rule it out the way a psychologist or psychiatrist would. Ideally, programs like this might help clinicians quickly find people who are struggling and need help. Right now, Reece says that patients wait several months on average between depression sets in and treatment. "If someone is depressed and really needs services, and they don't get that assessment, anything that can help them get assessed is good," he says.
And computers are able to look at vast amounts of social media information, something which a doctor could never do. Researchers have created computer scripts that can parse out signs of depression from Twitter, Facebook or Tumblr feeds. That means a program like this could provide useful information that doctors otherwise wouldn't have, says , a social and computer scientist at Georgia Tech who was not involved in this study. "The goal lies in empowering the clinician with tools that would help people manage or cope with these challenges above and beyond what they're currently using."
But we aren't quite there yet, says Dr. Megan Moreno, an adolescent psychiatrist at Seattle Children's Hospital who studies social media and mental illness. "We're probably still a ways away from having these types of approaches normalized as part of clinical care," she says. "I think this is a cool and interesting piece to the puzzle."
Actually using this information for depression screening comes with legal and privacy issues. "We're talking about the perspective that we could find people who need help. There's so much hope and optimism, but if we develop ways to monitor [social media data], then who else will monitor?" Moreno says. "Could these things come back and affect someone's employment or insurability? To some extent, people assume some of that is already happening."
Companies could already be collecting the data for this type of use. After all, Tweets, Tumblr and many Instagram posts are freely available to anyone with an Internet connection.
Copyright 2020 NPR. To see more, visit https://www.npr.org.