Associative Memories
Description
An associative
memory is a content-addressable structure that maps a set of input
patterns to a set of output patterns. There are two types of associative memory:
autoassociative and heteroassociative.
An autoassociative memory retrieves a previously stored pattern that most closely
resembles the current pattern. In a heteroassociative memory, the retrieved
pattern is, in general, different from the input pattern not only in content but
possibly also in type and format.
Linear
Associator
The linear
associator is one of the simplest and among the first studied associative
memory models. It is a feedforward type of network where the output is produced
in a single feedforward computation. It can be used as an autoassociator
as well as a heteroassociator. However, it possesses a very low memory capacity.
Hopfield
Model
The Hopfield
model is an autoassociative memory. Proposed by John Hopfield in 1982,
it is an ensemble of simple processing units that have a fairly complex collective
computational abilities and behavior. The Hopfield model is different from
the linear associator model in that it computes its output recursively in time
until the system becomes stable. Hopfield networks were originally designed using
bipolar units with no learning procedure to incrementally arrive at stable values
for the connection weights. Later versions of the model allowed for continuous-valued
units. Furthermore, a learning procedure was devised.
Bi-directional
Associative Memory
In 1988, Kosko extended
the Hopfield model by incorporating an additional layer to perform recurrent autoassociations
as well as heteroassociations on the stored memories. The network structure
of the bi-directional
associative memory (BAM) model is similar to that of the linear associator
but the connections are bi-directional, i.e. BAM allows forward and backward flow
of information between the layers. The BAM model can perform both autoassociative
and heteroassociative recall of stored information.
