Article of the Month – October 2021
October 1, 2021
Mayo Clinic
October 1, 2021Deep Learning
Slowly but surely, Deep Learning (DL) has become a significant part of our lives.(1) Since it was introduced in Science magazine in 2006, DL has been an important research topic in Artificial Intelligence (AI) and Machine Learning studies (ML).(2) DL is a subfield of ML that applies Artificial Neural Networks with multiple hidden processing layers that are trained to learn hierarchy and representations from large amounts of data in order to manage assignments of classification and recognition; all this is achieved using an unsupervised pre-training and a supervised fine-tuning approach.(2)
The multiple layers of representation are acquired by creating simple but non-linear modules. Each of those transforms the representation from the first level (beginning with the raw input) into a higher, more abstract level, allowing very complex functions to be learned. A vital aspect of DL is that models learn from data using general-purpose learning methods, and the layers of features are not planned by human engineers.(3)
The process resulting in a relationship between inputs and outputs is described as mapping with multiple hidden layers (figure 1). To achieve this, the more advanced DL models use backpropagation procedures while the simpler DL models use feedforward procedures.
DL has proven to be good at deciphering structures in high-dimensional data; thus, it can be applied in many fields. Some of the most significant applications of DL can be seen in areas like image recognition, speech recognition, self-driving cars, object detection, temporal data processing, cancer diagnosis, biomedicine, and several others.
In medicine-related research, DL models have helped predict the activity of potential drugs, analyzing particle accelerator data, reconstructing brain circuits, and predicting the effects of mutations in non-coding DNA on gene expression and disease.(1,3,4,5)
The process resulting in a relationship between inputs and outputs is described as mapping with multiple hidden layers (figure 1). To achieve this, the more advanced DL models use backpropagation procedures while the simpler DL models use feedforward procedures.
DL has proven to be good at deciphering structures in high-dimensional data; thus, it can be applied in many fields. Some of the most significant applications of DL can be seen in areas like image recognition, speech recognition, self-driving cars, object detection, temporal data processing, cancer diagnosis, biomedicine, and several others.(1,3,4,5)
In medicine-related research, DL models have helped predict the activity of potential drugs, analyzing particle accelerator data, reconstructing brain circuits, and predicting the effects of mutations in non-coding DNA on gene expression and disease.(3)
The process resulting in a relationship between inputs and outputs is described as mapping with multiple hidden layers (figure 1). To achieve this, the more advanced DL models use backpropagation procedures while the simpler DL models use feedforward procedures.
DL has proven to be good at deciphering structures in high-dimensional data; thus, it can be applied in many fields. Some of the most significant applications of DL can be seen in areas like image recognition, speech recognition, self-driving cars, object detection, temporal data processing, cancer diagnosis, biomedicine, and several others. In medicine-related research, DL models have helped predict the activity of potential drugs, analyzing particle accelerator data, reconstructing brain circuits, and predicting the effects of mutations in non-coding DNA on gene expression and disease.(1,3,4,5)
In medicine-related research, DL models have helped predict the activity of potential drugs, analyzing particle accelerator data, reconstructing brain circuits, and predicting the effects of mutations in non-coding DNA on gene expression and disease.(3)
There are plenty of types of DL architectures, but the most common ones are Convolutional Neural Network (CNN), (figure2). Recurrent Neural Network (RNN), and Long Short-Term Memory (LSTM). CNNs were mainly developed for large-scale image and video classification and recognition and were designed with three layers: convolutional, sampling/pooling, and fully connected. The convolutional and pooling layers resemble the classic examples of simple cells and complex cells in visual neuroscience, and the architecture mimics the hierarchy in the visual cortex ventral pathway. CNNs are excellent at detection, segmentation, and recognition of objects and sections in images.(1,2,3,5)
There are plenty of types of DL architectures, but the most common ones are Convolutional Neural Network (CNN), (figure2). Recurrent Neural Network (RNN), and Long Short-Term Memory (LSTM). CNNs were mainly developed for large-scale image and video classification and recognition and were designed with three layers: convolutional, sampling/pooling, and fully connected. The convolutional and pooling layers resemble the classic examples of simple cells and complex cells in visual neuroscience, and the architecture mimics the hierarchy in the visual cortex ventral pathway. CNNs are excellent at detection, segmentation, and recognition of objects and sections in images.(1,2,3,5)
RNNs are DL models that take time into account and are often involved in tasks with sequential inputs, like speech and language (figure 3). RNNs process the input sequence one element at a time while keeping in the hidden units a ‘state vector’ that contains the history of all the past elements of the sequence. RNNs have proven to be great at predicting the next word in a sentence or sequence. Long Short-Term Memory (LSTM) networks resemble RNNs because they use special hidden units that remember inputs for an extended period of time. They are widely used in speech recognition to go from acoustics to the sequence of characters in a text, not only relying on memorization but also ‘learned’ reasoning and symbol manipulation.(2,3)
These and many other models are developed using all the data available globally to optimize technology and everyday activities. Nevertheless, there is still much more to be discovered since the unceasing research is still working to accomplish understanding, comprehension, and further reasoning in Deep Learning.(4)
RNNs are DL models that take time into account and are often involved in tasks with sequential inputs, like speech and language (figure 3). RNNs process the input sequence one element at a time while keeping in the hidden units a ‘state vector’ that contains the history of all the past elements of the sequence. RNNs have proven to be great at predicting the next word in a sentence or sequence. Long Short-Term Memory (LSTM) networks resemble RNNs because they use special hidden units that remember inputs for an extended period of time.ulation.(2,3)
They are widely used in speech recognition to go from acoustics to the sequence of characters in a text, not only relying on memorization but also ‘learned’ reasoning and symbol manipulation.
These and many other models are developed using all the data available globally to optimize technology and everyday activities. Nevertheless, there is still much more to be discovered since the unceasing research is still working to accomplish understanding, comprehension, and further reasoning in Deep Learning.(3,4)
References