Artificial Neural Networks

The Animal Brain

Much is now known about the animal brain, but so much more have to be learned. The partitioning of the brain into a left and right hemisphere, and into various subcomponents such as the cerebrum, pituitary gland, thalamus, hypothalamus, hippocampus, cerebellum, hind brain, mid brain, are all known. Also, the general functions of the various regions of the human brain have been mapped out, e.g., the back regions for vision, the frontal lobe for logic and higher level thought processes, certain specific region for auditory perception, another region for somato-sensory perception, etc.

What are less known are the specific electro-chemical interactions among neurons and the various neurophysiological processes involved in short-term and long term memory, not to mention even vaguer concepts such as "consciousness" and "intuition".

There is a tight relationship between the weight and size of the brain and the intellectual capacity of the animal. The human brain is the largest among all animal brains. A chimpanzee's brain is big compared to that of other animals, regardless of their relative weight, but lacks some parts of the human brain, such as the frontal lobe, which is associated with man's logic and reasoning. Lower forms of animals in the animal kingdom structure that exhibit limited intelligence have smaller brains.

The cerebral cortex is the largest part of the brain and is composed of numerous interconnected neurons arranged in very thin sheets that are highly convoluted. There are approximately 1011 neurons in the human brain, and each is connected to a thousand to up to 10,000 other neurons. The amount of interconnectivity in the average human brain is therefore in the order of 1014 or 1015!

The brain is mainly composed of massively interconnected cells called neurons. A typical neuron has a cell body called the soma, and root-like structures through which input signals are received, called dendrites. These dendrites connect to other neurons through what is called a synapse. Signals collected through the numerous dendrites are accumulated in the soma. If the accumulated signal exceeds the neuron's threshold, then the cell is activated. This results in an electrical spike that is sent down the output channel called the axon. Otherwise, the cell remains inactive.

The efficiency by which signals from one neuron is sent to another neuron via a particular synapse is called the synaptic efficiency. Synapses are truly channels through which electrical impulses cross. These electrical impulses are regulated by chemicals known as neuro-transmitters.

The nature of the synaptic membranes and the chemical composition of neuro-transmitters determine whether a synapse is inhibitory or excitatory. When the synapse is excitatory, then the signal from the sending neuron tends to activate the neuron to which it is sent. Inhibitory synapses tend to prevent the activation of the neuron to which a signal is sent.

The strength of the electric signal that finally reaches the soma of the receiving neuron therefore depends on the strength of the original signal as well as on the efficiency of the synapse through which it will pass. The nature of the signal (inhibitory or excitatory) depends on the kind of neuro-transmitters found in the specific synaptic channel and on the type of synaptic membrane. In artificial neurons, we use real-valued connection weights to model the synaptic efficiency between two neurons, and their sign (positive/negative) indicate whether they are excitatory or inhibitory.

Modeling the Biological System

The artificial neural networks studied thus far in the literature have but a very, very small fraction of the number of neurons in the brain. Likewise, the interconnectivity of current artificial systems is a far cry from the massive interconnections in the brain. Thus, we have a long way to go in regard to modeling the animal brain, not to mention that we still do not have a full understanding of each of the several types of biological neurons found in the animal brain.

Neurobiological evidence seems to indicate that a given neuron is either inhibitory to all neurons to which it is connected or excitatory to all. There seems to be no evidence to support connection weights from one node to be excitatory towards some nodes and inhibitory to others. Note that in many artificial neural network models, a mixture of inhibitory and excitatory connections emanating from a single node is allowed.

There are other neurobiological data that are generally ignored in most artificial models. There are several types of neurons in the nervous system, while most models use a single type of node. Also, sizable clusters of neurons in the brain constitute “cortical columns”, which, when taken as a single logical unit, do not behave the way nodes in artificial neural networks do. As such, perhaps the nodes of current artificial systems should model these columns, instead of the single biological neuron.