One very brainy bird

For the study, each pigeon learned to discriminate cancerous from non-cancerous images and slides using traditional “operant conditioning,” a technique in which a bird was rewarded only when a correct selection was made; incorrect selections were not rewarded and prompted correction trials.

If pigeons went to medical school and specialized in pathology or radiology, they’d be pretty good at distinguishing between digitized microscope slides and mammograms of normal and cancerous breast tissue, according to a new study from the University of Iowa and the University of California, Davis.

With some training and selective food reinforcement, pigeons performed as well as humans in categorizing digitized slides and mammograms of benign and malignant human breast tissue, the researchers found. The pigeons were able to generalize what they had learned, so that when the researchers showed them a completely new set of normal and cancerous digitized slides, they correctly identified them. Their accuracy, like that of humans, was modestly affected by the presence or absence of color in the images, as well as by degrees of image compression. The pigeons also learned to correctly identify cancer-relevant micro calcifications on mammograms, but they had a tougher time classifying suspicious masses on mammograms—a task considered difficult even for skilled human observers, the authors note in the paper, published online Nov. 18 in the journal PLOS One.

Edward Wasserman

“These results go a long way toward establishing a profound link between humans and our animal kin,” says Edward Wasserman, professor of psychological and brain sciences at the UI and study co-author. “Even distant relatives—like people and pigeons—are adept at perceiving and categorizing the complex visual patterns that are presented in pathology and radiology images, surely a task for which nature has not specifically prepared us.”

The pigeons’ successes and difficulties provide a window into how physicians process visual cues present on slides and X-rays to diagnose and classify disease risk. This work also suggests that pigeons’ remarkable ability to discriminate between complex visual images could be put to good use as trained medical image observers, to help researchers explore image quality and the impact of color, contrast, brightness, and image compression artifacts on diagnostic performance.

Outstanding learners

Although a pigeon’s brain is no bigger than the tip of an index finger, it turns out that the neural pathways involved, including the basal ganglia and cortical-striatal synapses, operate in ways very similar to those at work in the human brain.

According to Wasserman, the common pigeon—or rock dove—(Columba livia) has a tremendous capacity to discriminate and categorize a wide range of objects and images.

“Research over the past 50 years has shown that pigeons can distinguish identities and emotional expressions on human faces, letters of the alphabet, misshapen pharmaceutical capsules, and even paintings by Monet vs. Picasso,” Wasserman says. “Their visual memory is equally impressive, with a proven recall of more than 1,800 images.”

The collaboration began when study co-author Richard Levenson, at UC Davis, learned about Wasserman’s earlier research on the visual short-term memory capacity of pigeons and people. Levenson wondered how pigeons would perform on pathology slides. 

Pigeons especially adept at discriminating among breast cancer slides

For the study, each pigeon learned to discriminate cancerous from non-cancerous images and slides using traditional “operant conditioning,” a technique in which a bird was rewarded only when a correct selection was made; incorrect selections were not rewarded and prompted correction trials. Training with stained pathology slides included a large set of benign and cancerous samples from routine cases at UC Davis Medical Center.

Some birds, for example, first learned to recognize benign or malignant samples in full color at low magnification (4X) and then progressed to medium (10X) and high (20X) magnifications. They were also tested using monochrome samples to eliminate color and brightness as potential cues, as well as samples with different levels of image compression, a procedure commonly used to reduce the size of digital data sets. To rule out the possibility that the birds were relying on rote memorization for the tests, brand-new samples were presented and food was dispensed regardless of whether the pigeons made a correct selection. And, indeed, the pigeons performed virtually as well on images that they had never been shown before, indicating that they had, in an extremely narrow sense, learned pathology.

“The birds were remarkably adept at discriminating between benign and malignant breast cancer slides at all magnifications, a task that can perplex inexperienced human observers, who typically require considerable training to attain mastery,” says Levenson, professor of pathology and laboratory medicine at UC Davis Health System. “Pigeons’ accuracy from day one of training at low magnification increased from 50 percent correct to nearly 85 percent correct at days 13 to 15.”

Wasserman, who has conducted studies on pigeons for over 40 years, found the pigeons especially adept at discerning pathology slides.

“The pigeons learned to discriminate benign from cancerous slides as fast in this research as in any other study we’ve conducted on pigeons in our laboratory,” Wasserman says. “In fact, when we showed a cohort of four birds a set of uncompressed images, an approach known as “flock-sourcing,” the group’s accuracy level reached an amazing 99 percent correct, higher than that achieved by any of the four individual birds.”

Density on mammograms a challenge for pigeons

For the mammogram study, the birds were trained to detect images with and without microcalcifications and to distinguish malignancy in breast masses using a similar process. Their accuracy averaged 84 percent for images with microcalcifications that they had been trained upon, and 72 percent for novel images—a level of performance on par with human radiologists and radiology residents who were given the same cases to review.

The birds, however, had difficulty evaluating the malignant potential of breast masses (without microcalcifications) detected on mammograms, a task the authors acknowledge is “very challenging.” Human radiologists achieved an accuracy rate of about 80 percent when viewing images of the relatively subtle masses used in this study. But, the pigeons took many weeks—instead of days that they had needed to master the histopathology tasks—to learn to classify the breast masses in the mammogram training set. More strikingly, after the training phase, when they were finally shown novel, previously unseen images, the birds utterly failed to perform at a level better than chance.

“The data suggest that the birds were just memorizing the masses in the training set, and never learned how to key in on stellate margins and other features of the lesions that can correlate with malignancy,” Levenson said. “But, as this task reflects the difficulty even humans have, it indicates how pigeons may be faithful mimics of the strengths and weaknesses of humans in viewing medical images.”

Pigeons as human surrogates?

After years of education and training, physicians can sometimes struggle with the interpretation of microscope slides and mammograms. Levenson, a pathologist who studies artificial intelligence for image analysis and other applications in biology and medicine, believes there is considerable room for enhancing the process.

“While new technologies are constantly being designed to enhance image acquisition, processing, and display, these potential advances need to be validated using trained observers to monitor quality and reliability,” Levenson said. “This is a difficult, time-consuming, and expensive process that requires the recruitment of clinicians as subjects for these relatively mundane tasks.

“Pigeons’ sensitivity to diagnostically salient features in medical images suggest that they can provide reliable feedback on many variables at play in the production, manipulation, and viewing of these diagnostically crucial tools, and can assist researchers and engineers as they continue to innovate.”

Other study authors include Victor Navarro at the UI and Elizabeth Krupinski, from Emory University.

This research was funded by grants from the National Institutes of Health (MH47313, EY019781, and MH076226).