Hybrid System Model

Hybrid neuro-symbolic systems (HNSS) or knowledge-based neurocomputing systems [Hilario97, Cloete00, Wemter00] are computational systems which are based on ANN’s but also allow symbolic interpretation, or interaction with symbolic components. The focus of these systems is mainly on methods to encode prior knowledge, and to extract, refine, and revise knowledge within a neurocomputing system. These systems allow us to integrate symbolic and connectionist approaches, exploiting the complementarities between these two knowledge representation and acquisition techniques.

Conventional symbolic AI is strongly based on symbol manipulation. The representation and manipulation of symbolically structured knowledge, coded into a language close to natural language, is exactly the central point which led to the development of the (symbolic) ES’s or KBS’s. The advantages of these systems lie in their ability to represent comprehensible knowledge.

On the other hand, ANN’s are based on a model of the human brain, and an attempt to model explicit symbol manipulation is largely absent. The internal knowledge representation of ANN’s (connection weights) can’t be easily interpreted, and they are known as “black boxes”. However, ANN’s are powerful tools used to learn/acquire knowledge from available data and to generalize the acquired knowledge. The ANNs can easily learn and handle inexact and uncertain information.

Since traditional symbolic ES’s and KBS are well-designed to handle expert knowledge represented by symbolic rules, and connectionist systems are powerful tools used to learn and generalize knowledge obtained from practical cases (including uncertain and inexact data), combinating these two approaches will explore their complementarities in order to improve the overall system performance.

Neuro-Symbolic Integration Strategies

Several attempts to integrate neural and symbolic processing were done in the 90’s. Roughly, we can distinguish three main architectural approaches to achieve neuro-symbolic integration:

• unified architectures
• translational architectures
• hybrid architectures

Neuro-Symbolic Integration Strategies

This classification dimension should be seen as a continuum where “unified” and “hybrid” approaches are the two opposite points.

In the unified approach, symbolic processing capabilities are modeled within the connectionist networks, i.e., unified strategies are premised on the claim that there is no need for symbolic structures and processes as such. Full symbol processing functionalities can emerge from neural processing. Typically, connectionist symbol processing (CSP) implement some high-level symbol processing, exhibiting properties like compositionality, access and manipulation of symbolic structures such lists, stacks, and trees. The systems DCPS [Touretzky88], BoltzCONS [Touretzky90], RAAM [Pollack90] and CONSYDERR [Sun91] are examples that use this approach. On the other side of unified approach, we find extended symbolic systems that integrate inside the symbolic inference engine connectionist properties like adaptive learning and weighted decisions.

The translational approach, also called transformational models [Medsker94], can be viewed as an intermediate class between the unified and hybrid approaches. Like unified models, they rely on neural networks as data processors, but they can start from or end with symbolic structures. Typically, their objective is to translate or transform symbol structures into neural networks before processing, or to extract symbolic structures from neural networks after processing [Hilario97]. Translational approaches are divided in two groups:

1. Symbolic-to-connectionist translation. Symbolic rules are converted into a set of connections and respective weights (knowledge insertion/compilation), obtaining a resulting neural network [Towell91, Giacometti92, Mahoney96, Setiono99, Osorio00, Cloete00]. This network should be equivalent to the original rule set, and we expect to obtain the same answers from both systems (symbolic rules/connectionist network) when  supplied with the same inputs. Different types of symbolic rules can be used within this approach: propositional rule, rules obtained from decision trees, fuzzy rules, rules with certainty-factor or probability ratings, and even deterministic finite state automata can be converted into neural networks.

2. Connectionist-to-symbolic translation. Weighted connections and network units are converted into a set of rules (knowledge extraction), using decompositional, pedagogical, or ecletic (mixed) methods [Giacommeti92, Fu94, Andrews95, Lu95, Cechin98, Cloete 00, Bologna00, Setiono00]. The connectionist to symbolic translation can also be done using different types of symbolic rules, usually the same types as in symbolic to connectionist translation.