News Menu

AI researchers ask: What鈥檚 going on inside the black box?

illustration of a neural network
Researchers can train artificial brain-like neural networks to classify images, such as cat pictures. Using a series of manipulated images, the scientists can figure out what part of the image鈥攕ay the whiskers鈥攊s used to identify it as a cat. However, when the same technology is applied to DNA, researchers are not certain what parts of the sequence are important to the neural net. This unknown decision process is known as a 鈥渂lack box鈥. Illustration: Ben Wigler

黑料吃瓜资源 (CSHL) Assistant Professor Peter Koo and collaborator Matt Ploenzke reported a way to train machines to predict the function of DNA sequences. They used 鈥渘eural nets,鈥 a type of artificial intelligence (AI) typically used to classify images. Teaching the neural net to predict the function of short stretches of DNA allowed it to work up to deciphering larger patterns. The researchers hope to analyze more complex DNA sequences that regulate gene activity critical to development and disease.

Machine-learning researchers can train a brain-like 鈥渘eural net鈥 computer to recognize objects, such as cats or airplanes, by showing it many images of each. Testing the success of training requires showing the machine a new picture of a cat or an airplane and seeing if it classifies it correctly. But, when researchers apply this technology to analyzing DNA patterns, they have a problem. Humans can鈥檛 recognize the patterns, so they may not be able to tell if the computer identifies the right thing. Neural nets learn and make decisions independently of their human programmers. Researchers refer to this hidden process as a 鈥渂lack box.鈥 It is hard to trust the machine鈥檚 outputs if we don鈥檛 know what is happening in the box.

Koo and his team fed DNA (genomic) sequences into a specific kind of neural network called a convolutional neural network (CNN), which resembles how animal brains process images. Koo says:

鈥淚t can be quite easy to interpret these neural networks because they鈥檒l just point to, let鈥檚 say, whiskers of a cat. And so that鈥檚 why it鈥檚 a cat versus an airplane. In genomics, it鈥檚 not so straightforward because genomic sequences aren鈥檛 in a form where humans really understand any of the patterns that these neural networks point to.鈥

Koo鈥檚 research, reported in the journal Nature Machine Intelligence, introduced a new method to teach important DNA patterns to one layer of his CNN. This allowed his neural network to build on the data to identify more complex patterns. Koo鈥檚 discovery makes it possible to peek inside the black box and identify some key features that lead to the computer鈥檚 decision-making process.

But Koo has a larger purpose in mind for the field of artificial intelligence. There are two ways to improve a neural net: interpretability and robustness. Interpretability refers to the ability of humans to decipher why machines give a certain prediction. The ability to produce an answer even with mistakes in the data is called robustness. Usually, researchers focus on one or the other. Koo says:

鈥淲hat my research is trying to do is bridge these two together because I don鈥檛 think they鈥檙e separate entities. I think that we get better interpretability if our models are more robust.鈥

Koo hopes that if a machine can find robust and interpretable DNA patterns related to gene regulation, it will help geneticists understand how mutations affect cancer and other diseases.

Written by: Jasmine Lee, Content Developer/Communicator | [email protected] | 516-367-8845


Funding

National Cancer Institute, Simons Center for Quantitative Biology, National Institutes of Health

Citation

Koo, P., et al., 鈥淚mproving representations of genomic sequence motifs in convolutional networks with exponential activations鈥, Nature Machine Intelligence, February 8, 2021. DOI:

Stay informed

Sign up for our newsletter to get the latest discoveries, upcoming events, videos, podcasts, and a news roundup delivered straight to your inbox every month.

  Newsletter Signup

Principal Investigator

Peter Koo

Peter Koo

Associate Professor
Cancer Center Member
Ph.D., Yale University, 2015

Tags