Purpose

The Fractal Neural Network (FNN) has been proposed to tackle the limitation of traditional models in representing higher cognitive functions.

The first argument is that higher order cognitive functions are not represented by simple network but by several sub-networks. Recent physiological findings suggest that higher cognitive functions always work based on lower sub-modules (see Fig. 1). Visual module contains many sub-modules such as "color", "motion" and "street names". Language module consists of lower order functions; "tool names", "animal names", and "verbs" [7] & [8].

The second argument is the fact that a network constructed from a large number of sub-networks based on traditional models results an explosion of information processing activities.This can happen due to traditional models deficiency in abstraction function [1].

These arguments are supported by other recent neuroanatomical and physiological evidences that suggest the existence of modularity and hierarchy in the cerebral cortex [9]. Hierarchical connections of several functional modules makes the whole the cerebral cortex. Each module, other than a partial set of a higher module, has also its own sub-modules Each module "abstracts" the learning results from its lower sub-modules and fetch them to the higher module for further learning.

The report is based
on following research
papers by
Takeshi Ieshima and
Akifumi Tokosumi :

1999
How could
neural networks
represent higher
cognitive functions?:
A computational model
based on a fractal
neural network,
ICCS/JCSS99

1999
Modularity and Hierarchy:
The Fractal Theory of
Consciousness
based on the
Fractal Neural Network,
Toward a Science of
Consciousness,
TOKYO '99

1999
Modularity and Hierarchy
in the Cerebral Cortex:
A Proposal of Fractal
Neural Network,
Neural Networks in
Applications NN '99,

Previous discussion implied Fractal Neural Network should have modularity and hierarchy. To accommodate those characteristic, we need to look at two general classes of neural networks; feed-forward and recurrent networks.

Within feed-forward networks, teaching signals flow uni-directionally and change connection weights with any processing signals. It implies some kind of hierarchy, without modularity. While in recurrent networks, without any teaching signals, neurons perform self-organising learning by changing connection weights using Hebb rule. It implies some sort of modularity, but no hierarchy [2] (See Fig. 2).

Although traditional recurrent networks can provide modularity, they would not be sufficient because their deficiency in abstraction capability. Fortunately a network model capable of "abstract" the module signals has been developed by Morita [6].

This model is called Nonmonotone Neural Networks (NNN). One NNN unit is built from a pair of excitatory and inhibitory neurons (See Fig. 3).

Fig. 3. Network Structure of Nonmonotone Neural Network

The dynamics of analog NNN follows this formula:

denotes the instantaneous potential of the i-th neuron, denotes the output and is a time constant. The nonmonotone dynamics is the results of using nonmonotonic output function (See Fig. 4)

Fig. 4. Output function of Nonmonotone Neural Network

The output function is given by

where c, and h are positive constants and is a parameter which is usually negative. Note if = 1, the dynamics are the same as conventional monotone dynamics in the analog Hopfield model [5].

Based on its nonmonotonic dynamics, NNN is able to respond like pattern-selective neurons by performing "abstraction" from input signals [3]. Then after some amount of learning NNN can act as a functional module and send this pattern-selective signals to the higher module.

With hierarchical structure of NNNs we can achieve both modularity and hierarchy and at the same time using NNN's abstraction capability to avoid explosion of information processing. This model is named Fractal Neural Network (FNN) [3]. It consists layers of modules (See Fig 1). Each NNN module consists of the same number of sub-modules and the highest module (output module) is a singular. Each module has also the same number of neurons and same nonmonotonic output function.

As mentioned above, FNN is built from many NNNs in form of hierarchical connection. Each module receives input signals from subordinate module. Then the module will regulate the connection weights between neurons using simple Hebb Rule, with higher thresholds value for higher layer neurons. This module will abstract the input signals and send it to superordinate module.

The lowest layer's modules receive input signals from external environment. After learning these modules will send the abstraction output to the higher module. Figure 1 shows how this FNN model works on cerebral cortex to provide higher cognitive function. Functional modules in sensory cortecies; such as color, edge and tone perform abstraction and send them to association cortecies. As the lower modules, association cortecies modules like "what", "when" and "who" send abstracted signal to the frontal association cortex. And finally motor cortex receive abstracted signals like "dangerous" or "my favourite" [3] (See Fig. 1).

As a model, FNN has characteristics of both feed-forward and recurrent networks. This property will open new interesting area in Neural Network Application.

FNN has also provided a model to represent higher cognitive function. The originator of FNN has also proposed a Theory of Consciousness based on FNN [2].

As an extension of NNN, FNN inherits the superiority of NNN over other traditional recurrent networks. NNN exhibits higher recollection ability and memory capacity compared with traditional models like Hopfield (Hopfield model has capacity of 0.138N, while NNN claims its memory capacity is at least 0.2N) [5].

The FNN itself is quite new and still very much in theoretical area. The author of this report hasn't found any paper on formal and thorough analysis of the properties of Fractal Neural Network.

From the biological relevance point of view, NNN is not biologically realistic, since the real neuron of brain doesn't have an output function in Figure 4 [2].

References

[1] Takeshi Ieshima and Akifumi Tokosumi, 1999, How could neural networks represent higher cognitive functions?: A computational model based on a fractal neural network The Second International Conference on Cognitive Science and The 16th Annual Meeting of the Japanese Cognitive Science Society Joint Conference (ICCS/JCSS99)

[4] Morita, M.,1996, Computational study on the neural mechanism of sequential pattern memory. Cognitive Brain Research, 5, 137-146.

[5] Morita, M., 1996, Memory and learning of sequential patterns by nonmonotone neural networks. Neural Networks, 9, 1477-1489.

[6] Morita, M.,1993, Associative memory with nonmonotone dynamics.
Neural Networks, 6, 115-126.

[7] Caramazza, A., Hills, A., Leek, E., and Miozzo, M., 1994, The Organization of Lexical Knowledge in the Brain: Evidence from Category- and Modality-specific Deficits. Mapping the mind, Cambridge University Press.

[8] Damasio, H., Grabowski, T. J., Tranel, D., Hichwa, R. D., & Damasio, A., 1996, A Neural Basis for Lexical Retrieval. Nature 380, 499-505.

[9] Takeda, T., and Ohgo, H., Matsuoka, T., Yanagisawa, E., Nakano, K., & Suzuki, M., 1990, Fractal Analysis of Cerebral Sulcal Patterns. Jpn. J. Physiol., 40, suppl., S158.

topˆ

Takeshi Ieshima
Research Papers Index

Masahiko Morita
Research Papers Index