Artifical intelligence has recently taken the field of microscopy by storm. A form of AI called convolutional neural networks, or CNNs, has especially been of interest because it can analyze images as accurately as humans can.

Convolutional neural networks can be trained to recognize and discover complex patterns in images. As with human vision, giving CNNs many example images and pointing out what features to pay attention to can teach the computer to recognize patterns of interest.

These patterns could include biological phenomena difficult to see by eye. For example, one research group was able to train CNNs to identify skin cancer more accurately than trained dermatologists. Even more recently, colleagues of mine were able to train CNNs to identify complex biological signatures such as cell type in microscopy images.

Building on this work, we developed a new technology called biomarker-optimized CNNs, or BO-CNNs, to identify cells that have died. First, we needed to teach the BO-CNN to distinguish between clearly dead and clearly alive cells. So we prepared a petri dish with mice neurons that were engineered to produce a nontoxic protein called a genetically encoded death indicator, or GEDI, that colored living cells green and dead cells yellow. The BO-CNN could easily learn that green meant "alive" and yellow meant "dead." But it was also learning other features distinguishing living and dead cells that aren't so obvious to the human eye.

After the BO-CNN learned how to identify the characteristics that distinguished the green cells from the yellow, we showed it neurons that weren't distinguished by color. The BO-CNN was able to correctly label live and dead cells significantly faster and more accurately than people trained to do the same thing. The model could even look at images of cell types it had not seen before taken from different types of microscopes and still correctly identify dead cells.