Detecting Pancreatic Cancer Using Selfies

Perhaps all that narcissistic picture taking of oneself, which is seemingly so popular with smartphone users these days, can be put to effective use within the medical community, as researchers at the University of Washington (UW) have developed an app that could allow people to easily screen for pancreatic cancer and other diseases just by snapping a selfie.

The new app, described in a paper published recently in the Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies in an article entitled “BiliScreen: Smartphone-Based Scleral Jaundice Monitoring for Liver and Pancreatic Disorders”, and is set to be presented to the public at the upcoming Ubicomp 2017, the Association for Computing Machinery’s International Joint Conference on Pervasive and Ubiquitous Computing.

Pancreatic cancer has one of the worst prognoses, with a five-year survival rate of only 9%, in part because there are few revealing symptoms or noninvasive screening tools to catch a tumor before it spreads. One of the earliest symptoms of pancreatic cancer, as well as other diseases, is jaundice, a yellow discoloration of the skin and eyes caused by a buildup of bilirubin in the blood. The ability to detect signs of jaundice when bilirubin levels are minimally elevated, but before they’re visible to the naked eye, could enable an entirely new screening program for at-risk individuals.

“The problem with pancreatic cancer is that by the time you’re symptomatic, it’s frequently too late,” explained lead author Alex Mariakakis, a doctoral student at the Paul G. Allen School of Computer Science & Engineering at UW. “The hope is that if people can do this simple test once a month—in the privacy of their own homes—some might catch the disease early enough to undergo treatment that could save their lives.”

The new app, dubbed BiliScreen, uses a smartphone camera, computer vision algorithms, and machine-learning tools to detect increased bilirubin levels in a person’s sclera, or the white part of the eye. In an initial clinical study of 70 people, the BiliScreen app, used in conjunction with a 3D printed box that controls the eye’s exposure to light, correctly identified cases of concern 89.7% of the time, compared to the blood test currently used.

“The eyes are a really interesting gateway into the body—tears can tell you how much glucose you have, sclera can tell you how much bilirubin is in your blood,” noted senior study investigator Shwetak Patel, Ph.D., professor of computer science and engineering and electrical engineering at UW. “Our question was: Could we capture some of these changes that might lead to earlier detection with a selfie?”

Interestingly, the BiliScreen app builds on earlier work from the UW’s Ubiquitous Computing Lab, which previously developed BiliCam, a smartphone app that screens for newborn jaundice by taking a picture of a baby’s skin. A recent study in the journal Pediatrics showed BiliCam provided accurate estimates of bilirubin levels in 530 infants.

The blood test that doctors currently use to measure bilirubin levels, which is typically not administered to adults unless there is reason for concern, requires access to a health care professional and is inconvenient for frequent screening. BiliScreen was designed to be an easy-to-use, noninvasive tool that could help determine whether someone ought to consult a doctor for further testing. Beyond diagnosis, BiliScreen could also potentially ease the burden on patients with pancreatic cancer who require frequent bilirubin monitoring.

BiliScreen uses a smartphone’s built-in camera and flash to collect pictures of a person’s eye as they snap a selfie. The team developed a computer vision system to automatically and effectively isolate the white parts of the eye, which is a valuable tool for medical diagnostics. The app then calculates the color information from the sclera—based on the wavelengths of light that are being reflected and absorbed—and correlates it with bilirubin levels using machine-learning algorithms.

To account for different lighting conditions, the team tested BiliScreen with two different accessories: paper glasses printed with colored squares to help calibrate color and a 3-D printed box that blocks out ambient lighting. Using the app with the box accessory—reminiscent of a Google Cardboard headset—led to slightly better results.

“This relatively small initial study shows the technology has promise,” remarked study co-author Jim Taylor, M.D., professor in the department of pediatrics at UW.

Looking toward the future, the research team is hoping to test the app on a wider range of people at risk for jaundice and underlying conditions, as well as continuing to make usability improvements—including removing the need for accessories like the box and glasses.

“Pancreatic cancer is a terrible disease with no effective screening right now,” Dr. Taylor concluded. “Our goal is to have more people who are unfortunate enough to get pancreatic cancer to be fortunate enough to catch it in time to have surgery that gives them a better chance of survival.”