Deep Learning

Deep Learning is cutting-edge topic in the world of Machine Learning and AI. This post will provide basic information about Deep Learning. This includes answers to the below questions

1. How does human brain understand, process and learn that a human eye visualize?
2. What are limitation of regular computer programs?
3. What is Neural Networks/Deep Learning? What kind of problems does Deep Learning solve?

First, let’s understand what the visual cortex of the human brain do.

Visual Cortex of Human Brain:

The visual cortex of the brain is the area of the cerebral cortex that processes visual information received from the eyes. Complex mechanisms from retina to different visual areas allow us to read these lines. The visual system is inevitable for the way we interact with our surroundings as majority of our impressions, memories, feelings are bound to the visual perception. Visual area constitutes about 25 % of the cortex in humans with approximately 5 billion neurons. The study of the visual cortex has revealed many of these visual regions such as V1, V2, V3, V4 and MT on the basis of their anatomical architecture, topography and physiological properties [1]. These regions are involved in processing of multitude of informations (shape, orientation, color, movement, size etc) resulting from the visual pathways, thus making up an image applied to retina.

Fig 1. Organization of Visual System

Scientists and biologists discovered thirty different cortical areas that contribute to visual perception. The primary areas (V1) and secondary areas (V2) are surrounded by many other tertiary or associative visual areas such as V3, V4, V5 (or MT) involved in processing various attributes of trigger features [2, 3]. Areas V3 and V3A are selective to the form of stimuli [4], and neurons of area V4 are selective to colors [5]. Area V5 or MT (middle temporal) is an area where majority of cells are sensitive to motion and direction, and none of which are selective to color [6].

The parallel organization of visual system is involved in the establishment of two major visual pathways: Ventral and dorsal pathways which are indispensable for the object recognition [7, 8]. Fig 1 illustrates the parallel organization of visual system, two major pathways: Green part corresponds to the ventral pathway in the cortex ending in the temporal lobe [7, 9]. It is involved in the processing of information on the characteristics of the objects (shapes, colors, materials), that is, object recognition including faces. Orange part corresponds to the dorsal pathway in the cortex ending in the parietal lobe [7, 9]. This path is associated with spatial vision (action/location) of objects, and is involved in processing of action in space.

Computers:

Computers and computing help us achieve more complex goals than a human alone. Despite, there are many goals that are beyond the scope of computing because of one major limiting factor, computers could only follow the specific instructions they were given. Solving problems with programming requires writing specific step-by-step instructions for a computer to follow. We call these steps; algorithms. This constraint limited the type of problems where computers could help us to those where we:

1. Understand how to solve the problem
2. Can describe the solution with clear step-by-step instructions that a computer can understand

Let’s address this limitation by attempting to write a traditional program to solve an object detection problem. What steps will you write to detect a red car from a fleet of multicolored vehicles?

Neural Network:

Limitation of traditional computer program and the structure of the human brain inspires a Neural Network to solve classification and detection problems. Neural Network is essentially a Machine Learning model (more precisely, Deep Learning) that is used in unsupervised learning. A Neural Network is a web of interconnected entities known as nodes wherein each node is responsible for a simple computation. In this way, a Neural Network functions similarly to the neurons in the human brain. More precisely Deep Learning allows computer to learn from examples.

Solving problems with deep learning requires identifying some pattern, finding examples that highlight both sides of the pattern (the input and the output), and then letting a “neural network” learn the map between the two. This opens the types of problems where computers can help us to those where we have:

1. Identified a pattern within a problem
2. Data that exemplifies the pattern

Once those conditions are met, the next step would be to build the skills and workflows to solve problems with deep learning. In the forth coming posts, I’ll describe how to solve the some specific problems using Deep Learning.

References

1. Adaptation and Neuronal Network in Visual Cortex By Lyes Bachatene, Vishal Bharmauria and Stéphane Molotchnikoff. DOI: 10.5772/46011
2. Felleman DJ, Van Essen DC 1991 Distributed Hierarchical Processing in the Primate Cerebral Cortex. Cereb Cortex. 1 147
3. 1990, DeYoe EA, Olavarria J, Knierim J (1990) Modular and Hierarchical Organization of Extrastriate Visual Cortex in the Macaque Monkey. Cold Spring Harb Symp Quant Biol. 55 679696
4. Essen DC, Zeki SM 1978 The Topographic Organization of Rhesus Monkey Prestriate Cortex. J Physiol. 277 193226 .
5. 1992, Gadotti A, Cowey A (1992) Cortical Area 4 and its Role in the Perception of Color. J Neurosci. 12: 4056 EOF65 EOF.
6. Born RT, Bradley DC 2005 Structure and Function of Visual Area MT. Annu Rev Neurosci. 28 157189
7. MishkinM.UngerleiderL.Macko 1983 (1983) Object Vision and Spatial Vision: Two Cortical Pathways.Trends in Neuroscience. 6 414417 .
8. Di CarloJ. J.ZoccolanD.RustN. C. 2012 How Does the Brain Solve Visual Object Recognition? Neuron. 73 415434 .
9. MilnerD. A.GoodaleM.(1995 (1995).The visual brain in action. Oxford University Press