Inference in PL and FOL

Chapters 7, 8 and 9

+ Prolog Redux

Outline: PL Inference

Enumerative methods

Resolution in CNF

Sound and Complete

Forward and Backward Chaining using Modus Ponens in Horn Form

Sound and Complete

Proof methods

Proof methods divide into (roughly) two kinds:

Application of inference rules

Legitimate (sound) generation of new sentences from old

Proof = a sequence of inference rule applications
Can use inference rules as operators in a standard search algorithm

Typically require transformation of sentences into a normal form

Model checking

truth table enumeration (always exponential in n)

improved backtracking, e.g., Davis-Putnam-Logemann-Loveland (DPLL)

heuristic search in model space (sound but incomplete)

e.g., min-conflicts like hill-climbing algorithms

Efficient propositional inference

Two families of efficient algorithms for propositional inference:

Complete backtracking search algorithms

DPLL algorithm (Davis, Putnam, Logemann, Loveland)

Incomplete local search algorithms

WalkSAT algorithm

The DPLL algorithm

Determine if an input propositional logic sentence (in CNF) is satisfiable.

Improvements over truth table enumeration:

Early termination

A clause is true if any literal is true.

A sentence is false if any clause is false.

Pure symbol heuristic

Pure symbol: always appears with the same "sign" in all clauses.

e.g., In the three clauses (A Ú ØB), (ØB Ú ØC), (C Ú A), A and B are pure, C is impure.

Make a pure symbol literal true.

Unit clause heuristic

Unit clause: only one literal in the clause

The only literal in a unit clause must be true.

The DPLL algorithm

The WalkSAT algorithm

Incomplete, local search algorithm

Evaluation function: The min-conflict heuristic of minimizing the number of unsatisfied clauses

Balance between greediness and randomness

The WalkSAT algorithm

Hard satisfiability problems

Consider random 3-CNF sentences. e.g.,

(ØD Ú ØB Ú C) Ù (B Ú ØA Ú ØC) Ù (ØC Ú ØB Ú E) Ù (E Ú ØD Ú B) Ù (B Ú E Ú ØC)

m = number of clauses

n = number of symbols

Hard problems seem to cluster near m/n = 4.3 (critical point)

Hard satisfiability problems

Hard satisfiability problems

Median runtime for 100 satisfiable random 3-CNF sentences, n = 50

Resolution

Conjunctive Normal Form (CNF)

conjunction of disjunctions of literals

clauses

E.g., (A Ú ØB) Ù (B Ú ØC Ú ØD)

Resolution inference rule (for CNF):

l_i Ú… Ú l_k, m₁ Ú … Ú m_n

l_i Ú … Ú l_i-1Úl_i+1Ú … Ú l_k Ú m₁ Ú … Ú m_j-1Ú m_j+1 Ú... Ú m_n

where l_i and m_j are complementary literals.

E.g., P_1,3 Ú P_2,2, ØP_2,2

P_1,3

Resolution is sound and complete
for propositional logic

Resolution example

KB = (B_1,1 Û (P_1,2Ú P_2,1)) ÙØ B_1,1

α = ØP_1,2(negate the premise for proof by refutation)

The power of false

Given: (P) Ù (ØP)

Prove: Z

Can we prove ØZ using the givens above?

Applying inference rules

Equivalent to a search problem

KB state = node

Inference rule application = edge

Inference

Define: KB ├_iα = sentence α can be derived from KB by procedure i

Soundness: i is sound if whenever KB ├_iα, it is also true that KB╞ α

Completeness: i is complete if whenever KB╞ α, it is also true that KB ├_iα

Preview: we will define a logic (first-order logic) which is expressive enough to say almost anything of interest, and for which there exists a sound and complete inference procedure.

That is, the procedure will answer any question whose answer follows from what is known by the KB.

Completeness

Completeness: i is complete if whenever KB╞ α, it is also true that KB ├_i α

An incomplete inference algorithm cannot reach all possible conclusions

Equivalent to completeness in search (chapter 3)

Resolution

Soundness of resolution inference rule:

Ø(l_i Ú … Ú l_i-1Úl_i+1Ú … Ú l_k) Þ l_i

Øm_j Þ (m₁ Ú … Ú m_j-1Ú m_j+1 Ú... Ú m_n)

Ø(l_i Ú … Ú l_i-1Úl_i+1Ú … Ú l_k) Þ (m₁ Ú … Ú m_j-1Ú m_j+1 Ú... Ú m_n)

where l_i and m_j are complementary literals.

What if l_i and Øm_jare false?

What if l_i and Øm_j are true?

Completeness of Resolution

That is, that resolution can decide the truth value of S

S = set of clauses

RC(S) = Resolution closure of S = Set of all clauses that can be derived from S by the resolution inference rule.

RC(S) has finite cardinality (finite number of symbols P₁, P₂, … P_k), thus resolution refutation must terminate.

Completeness of Resolution (cont)

Ground resolution theorem = if S unsatisfiable, RC(S) contains empty clause.

Prove by proving contrapositive:

i.e., if RC(S) doesn’t contain empty clause, S is satisfiable

Do this by constructing a model:

For each P_i, if there is a clause in RC(S) containing ØP_iand all other literals in the clause are false, assign P_i = false

Otherwise P_i = true

This assignment of P_iis a model for S.

Other Reasoning Patterns

Given(s)

Conclusion

A Þ B, A

B

B Ù A

A

Rules that allow us to introduce new propositions while preserving truth values: logically equivalent

Two Examples:

Modus Ponens

And Elimination

Forward and backward chaining

Horn Form (restricted)

KB = conjunction of Horn clauses

Horn clause =

proposition symbol; or

(conjunction of symbols) Þ symbol

E.g., C Ù (B Þ A) Ù (C Ù D Þ B)

Modus Ponens (for Horn Form): complete for Horn KBs

α₁, … ,α_n, α₁ Ù … Ù α_n Þ β

β

Can be used with forward chaining or backward chaining.

These algorithms are very natural and run in linear time

Forward chaining

Idea: fire any rule whose premises are satisfied in the KB,

add its conclusion to the KB, until query is found

Forward chaining algorithm

Forward chaining is sound and complete for Horn KB

Forward chaining example

Proof of completeness

FC derives every atomic sentence that is entailed by KB (only for clauses in Horn form)

FC reaches a fixed point (the deductive closure) where no new atomic sentences are derived

Consider the final state as a model m, assigning true/false to symbols

Every clause in the original KB is true in m

a₁Ù … Ù a_k_Þb

Hence m is a model of KB

If KB╞ q, q is true in every model of KB, including m

Backward chaining example

Inference in first-order logic

Chapter 9

Outline

Reducing first-order inference to propositional inference

Unification

Generalized Modus Ponens

Forward chaining

Backward chaining

Resolution

Universal instantiation (UI)

Every instantiation of a universally quantified sentence is entailed by it:

"v α
Subst({v/g}, α)

for any variable v and ground term g

E.g., "x King(x) Ù Greedy(x) Þ Evil(x) yields:

King(John) Ù Greedy(John) Þ Evil(John)

King(Richard) Ù Greedy(Richard) Þ Evil(Richard)

King(Father(John)) Ù Greedy(Father(John)) Þ Evil(Father(John))

.

.

.

Existential instantiation (EI)

For any sentence α, variable v, and constant symbol k that does not appear elsewhere in the knowledge base:

$v α

Subst({v/k}, α)

E.g., $x Crown(x) Ù OnHead(x,John) yields:

Crown(C₁) Ù OnHead(C₁,John)

provided C₁ is a new constant symbol, called a Skolem constant

Reduction to propositional inference

Suppose the KB contains just the following:

"x King(x) Ù Greedy(x) Þ Evil(x)

King(John)

Greedy(John)

Brother(Richard,John)

Instantiating the universal sentence in all possible ways, we have:

King(John) Ù Greedy(John) Þ Evil(John)

King(Richard) Ù Greedy(Richard) Þ Evil(Richard)

King(John)

Greedy(John)

Brother(Richard,John)

The new KB is propositionalized: proposition symbols are

King(John), Greedy(John), Evil(John), King(Richard), etc.

Reduction contd.

Every FOL KB can be propositionalized so as to preserve entailment

(A ground sentence is entailed by new KB iff entailed by original KB)

Idea: propositionalize KB and query, apply resolution, return result

Problem: with function symbols, there are infinitely many ground terms,

e.g., Father(Father(Father(John)))

Reduction con’td.

Theorem: Herbrand (1930). If a sentence α is entailed by an FOL KB, it is entailed by a finite subset of the propositionalized KB

Idea: For n = 0 to ∞ do

create a propositional KB by instantiating with depth-n terms

see if α is entailed by this KB

Problem: works if α is entailed, loops if α is not entailed

Theorem: Turing (1936), Church (1936) Entailment for FOL is
semi-decidable (algorithms exist that say yes to every entailed sentence, but no algorithm exists that also says no to every non-entailed sentence.)

Problems with propositionalization

Propositionalization seems to generate lots of irrelevant sentences.

E.g., from:

"x King(x) Ù Greedy(x) Þ Evil(x)

King(John)

"y Greedy(y)

Brother(Richard,John)

it seems obvious that Evil(John), but propositionalization produces lots of facts such as Greedy(Richard) that are irrelevant

With p k-ary predicates and n constants, there are p·n^k instantiations.

Unification

We can get the inference immediately if we can find a substitution θ such that King(x) and Greedy(x) match King(John) and Greedy(y)

θ = {x/John,y/John} works

Unify(α,β) = θ if αθ = βθ

p q θ

Knows(John,x) Knows(John,Jane) {x/Jane}}

Knows(John,x) Knows(y,OJ) {x/OJ,y/John}}

Knows(John,x) Knows(y,Mother(y)) {y/John,x/Mother(John)}}

Knows(John,x) Knows(x,OJ) {fail}

Standardizing apart eliminates overlap of variables, e.g., Knows(z₁₇,OJ)

Unification

To unify Knows(John,x) and Knows(y,z),

θ = {y/John, x/z } or θ = {y/John, x/John, z/John}

The first unifier is more general than the second.

There is a single most general unifier (MGU) that is unique up to renaming of variables.

MGU = { y/John, x/z }

The unification algorithm

Generalized Modus Ponens (GMP)

p₁', p₂', … , p_n', ( p₁ Ù p₂ Ù … Ù p_n Þq)

qθ

p₁' is King(John) p₁ is King(x)

p₂' is Greedy(y) p₂is Greedy(x)

θ is {x/John,y/John} q is Evil(x)

q θ is Evil(John)

GMP used with KB of definite clauses (exactly one positive literal)

All variables assumed universally quantified

Soundness of GMP

Need to show that

p₁', …, p_n', (p₁ Ù … Ù p_n Þ q) ╞ qθ

provided that p_i'θ = p_iθ for all I

Lemma: For any sentence p, we have p ╞ pθ by UI

(p₁ Ù … Ù p_n Þ q) ╞ (p₁ Ù … Ù p_n Þ q)θ = (p₁θ Ù … Ù p_nθ Þ qθ)

p₁', …, p_n' ╞ p₁' Ù … Ù p_n' ╞ p₁'θ Ù … Ù p_n'θ

From 1 and 2, qθ follows by ordinary Modus Ponens

Example knowledge base

The law says that it is a crime for an American to sell weapons to hostile nations. The country Nono, an enemy of America, has some missiles, and all of its missiles were sold to it by Colonel West, who is American.

Prove that Col. West is a criminal

Example knowledge base contd.

... it is a crime for an American to sell weapons to hostile nations:

American(x) Ù Weapon(y) Ù Sells(x,y,z) Ù Hostile(z) Þ Criminal(x)

Nono … has some missiles, i.e., $x Owns(Nono,x) Ù Missile(x):

Owns(Nono,M₁) and Missile(M₁)

… all of its missiles were sold to it by Colonel West

Missile(x) Ù Owns(Nono,x) Þ Sells(West,x,Nono)

Missiles are weapons:

Missile(x) Þ Weapon(x)

An enemy of America counts as "hostile“:

Enemy(x,America) Þ Hostile(x)

West, who is American …

American(West)

The country Nono, an enemy of America …

Enemy(Nono,America)

Forward chaining algorithm

Forward chaining proof

Properties of forward chaining

Sound and complete for first-order definite clauses

Datalog = first-order definite clauses + no functions

FC terminates for Datalog in finite number of iterations

May not terminate in general if α is not entailed

This is unavoidable: entailment with definite clauses is semidecidable

Efficiency of forward chaining

Incremental forward chaining: no need to match a rule on iteration k if a premise wasn't added on iteration k-1

Þ match each rule whose premise contains a newly added positive literal

Matching itself can be expensive:

Database indexing allows O(1) retrieval of known facts

e.g., query Missile(x) retrieves Missile(M₁)

Forward chaining is widely used in deductive databases

Backward chaining algorithm

SUBST(COMPOSE(θ₁, θ₂), p) = SUBST(θ₂, SUBST(θ₁, p))

Backward chaining example

Prolog Inference

Q: which model do you think Prolog uses for inference?

Properties of backward chaining

Depth-first recursive proof search: space is linear w.r.t. size of proof

Incomplete due to infinite loops

Þ fix by checking current goal against every goal on stack

Inefficient due to repeated subgoals (both success and failure)

Þ fix using caching of previous results (extra space)

Prolog Execution

Prolog needs to choose which goal to pursue first, although logically it doesn’t matter. Why?

Treats goals in order, leftmost first.

A :- B,C,D.

B :- E,F.

-? A.

B is tried first, then C, then D.

E and F are pushed onto the stack, before C and D. Why?

Prolog Execution

Prolog also needs to choose which clause to pursue first.

Treats clauses in order, top-most first.

G.

A :- B,C,D.

B :- E,F.

B :- G.

To satisfy goal B, prolog tries E,F before G.

Procedural Prolog Programming

Order of Prolog clauses and goals crucial, can affect running times immensely

Order of goals tell which get executed first

Order of clauses tell which control branches are tried first.

A Singaporean example

likes(hari,X) :- makan(X), consumes(hari,X).

likes(min,X) :- likes(hari,X).

makan(meeSiam).

makan(rojak).

minum(rootBeerFloat).

consumes(hari,meeSiam).

Summary

Whew! That was a loooooooong lecture. What did we learn?

Enumeration: DPLL rules are similar to CSP heuristics.

Resolution is proof by refutation, used in PL.

Other forms of reasoning: Modus Ponens which requires Horn form.

FOL uses unification to find solutions, requires Skolem constants and functions.

Forward (undirected) and Backward (directed) chaining patterns to apply an inference mechanism.


	Enumerative methods
	Resolution in CNF
		Sound and Complete
	Forward and Backward Chaining using Modus Ponens in Horn Form
		Sound and Complete


Proof methods divide into (roughly) two kinds:

	Application of inference rules
		Legitimate (sound) generation of new sentences from old
		Proof = a sequence of inference rule applications Can use inference rules as operators in a standard search algorithm
		Typically require transformation of sentences into a normal form

	Model checking
		truth table enumeration (always exponential in n)
		improved backtracking, e.g., Davis-Putnam-Logemann-Loveland (DPLL)
		heuristic search in model space (sound but incomplete)
		e.g., min-conflicts like hill-climbing algorithms


	Two families of efficient algorithms for propositional inference:

	Complete backtracking search algorithms
	DPLL algorithm (Davis, Putnam, Logemann, Loveland)
	Incomplete local search algorithms
		WalkSAT algorithm


Determine if an input propositional logic sentence (in CNF) is satisfiable.

Improvements over truth table enumeration:
	Early termination
		A clause is true if any literal is true.
		A sentence is false if any clause is false.

	Pure symbol heuristic
		Pure symbol: always appears with the same "sign" in all clauses.
		e.g., In the three clauses (A Ú ØB), (ØB Ú ØC), (C Ú A), A and B are pure, C is impure.
		Make a pure symbol literal true.

	Unit clause heuristic
		Unit clause: only one literal in the clause
		The only literal in a unit clause must be true.


	Incomplete, local search algorithm
	Evaluation function: The min-conflict heuristic of minimizing the number of unsatisfied clauses
	Balance between greediness and randomness


	Consider random 3-CNF sentences. e.g.,
	(ØD Ú ØB Ú C) Ù (B Ú ØA Ú ØC) Ù (ØC Ú ØB Ú E) Ù (E Ú ØD Ú B) Ù (B Ú E Ú ØC)

		m = number of clauses
		n = number of symbols

		Hard problems seem to cluster near m/n = 4.3 (critical point)


	Median runtime for 100 satisfiable random 3-CNF sentences, n = 50


	Conjunctive Normal Form (CNF)
		conjunction of disjunctions of literals
		clauses
		E.g., (A Ú ØB) Ù (B Ú ØC Ú ØD)

	Resolution inference rule (for CNF):
	l_i Ú… Ú l_k, m₁ Ú … Ú m_n
	l_i Ú … Ú l_i-1Úl_i+1Ú … Ú l_k Ú m₁ Ú … Ú m_j-1Ú m_j+1 Ú... Ú m_n

	where l_i and m_j are complementary literals.
	E.g., P_1,3 Ú P_2,2, ØP_2,2
	P_1,3

	Resolution is sound and complete for propositional logic


	KB = (B_1,1 Û (P_1,2Ú P_2,1)) ÙØ B_1,1
	α = ØP_1,2(negate the premise for proof by refutation)


	Given: (P) Ù (ØP)
	Prove: Z



	Can we prove ØZ using the givens above?


	Equivalent to a search problem

	KB state = node
	Inference rule application = edge


	Define: KB ├_iα = sentence α can be derived from KB by procedure i
	Soundness: i is sound if whenever KB ├_iα, it is also true that KB╞ α
	Completeness: i is complete if whenever KB╞ α, it is also true that KB ├_iα
	Preview: we will define a logic (first-order logic) which is expressive enough to say almost anything of interest, and for which there exists a sound and complete inference procedure.
	That is, the procedure will answer any question whose answer follows from what is known by the KB.


	Completeness: i is complete if whenever KB╞ α, it is also true that KB ├_i α

	An incomplete inference algorithm cannot reach all possible conclusions
		Equivalent to completeness in search (chapter 3)


	Soundness of resolution inference rule:

	Ø(l_i Ú … Ú l_i-1Úl_i+1Ú … Ú l_k) Þ l_i
	Øm_j Þ (m₁ Ú … Ú m_j-1Ú m_j+1 Ú... Ú m_n)
	Ø(l_i Ú … Ú l_i-1Úl_i+1Ú … Ú l_k) Þ (m₁ Ú … Ú m_j-1Ú m_j+1 Ú... Ú m_n)

	where l_i and m_j are complementary literals.

	What if l_i and Øm_jare false?
	What if l_i and Øm_j are true?


	That is, that resolution can decide the truth value of S

	S = set of clauses
	RC(S) = Resolution closure of S = Set of all clauses that can be derived from S by the resolution inference rule.
	RC(S) has finite cardinality (finite number of symbols P₁, P₂, … P_k), thus resolution refutation must terminate.


Ground resolution theorem = if S unsatisfiable, RC(S) contains empty clause.
Prove by proving contrapositive:
	i.e., if RC(S) doesn’t contain empty clause, S is satisfiable
	Do this by constructing a model:
		For each P_i, if there is a clause in RC(S) containing ØP_iand all other literals in the clause are false, assign P_i = false
		Otherwise P_i = true
	This assignment of P_iis a model for S.


	Given(s)
	Conclusion


	A Þ B, A
	B

	B Ù A
	A

	Rules that allow us to introduce new propositions while preserving truth values: logically equivalent

	Two Examples:
	Modus Ponens

	And Elimination


Horn Form (restricted)
	KB = conjunction of Horn clauses
	Horn clause =
		proposition symbol; or
		(conjunction of symbols) Þ symbol
	E.g., C Ù (B Þ A) Ù (C Ù D Þ B)
Modus Ponens (for Horn Form): complete for Horn KBs
α₁, … ,α_n, α₁ Ù … Ù α_n Þ β
β

Can be used with forward chaining or backward chaining.
These algorithms are very natural and run in linear time


	Idea: fire any rule whose premises are satisfied in the KB,
		add its conclusion to the KB, until query is found


FC derives every atomic sentence that is entailed by KB (only for clauses in Horn form)
	FC reaches a fixed point (the deductive closure) where no new atomic sentences are derived
	Consider the final state as a model m, assigning true/false to symbols
	Every clause in the original KB is true in m
		a₁Ù … Ù a_k_Þb
	Hence m is a model of KB
	If KB╞ q, q is true in every model of KB, including m


	Reducing first-order inference to propositional inference
	Unification
	Generalized Modus Ponens
	Forward chaining
	Backward chaining
	Resolution


	Every instantiation of a universally quantified sentence is entailed by it:
	"v α Subst({v/g}, α)
	for any variable v and ground term g

	E.g., "x King(x) Ù Greedy(x) Þ Evil(x) yields:
		King(John) Ù Greedy(John) Þ Evil(John)
		King(Richard) Ù Greedy(Richard) Þ Evil(Richard)
		King(Father(John)) Ù Greedy(Father(John)) Þ Evil(Father(John))
		.
		.
		.


	For any sentence α, variable v, and constant symbol k that does not appear elsewhere in the knowledge base:
	$v α
	Subst({v/k}, α)

	E.g., $x Crown(x) Ù OnHead(x,John) yields:

	Crown(C₁) Ù OnHead(C₁,John)

	provided C₁ is a new constant symbol, called a Skolem constant


	Suppose the KB contains just the following:
		"x King(x) Ù Greedy(x) Þ Evil(x)
		King(John)
		Greedy(John)
		Brother(Richard,John)

	Instantiating the universal sentence in all possible ways, we have:
		King(John) Ù Greedy(John) Þ Evil(John)
		King(Richard) Ù Greedy(Richard) Þ Evil(Richard)
		King(John)
		Greedy(John)
		Brother(Richard,John)

	The new KB is propositionalized: proposition symbols are

		King(John), Greedy(John), Evil(John), King(Richard), etc.


	Every FOL KB can be propositionalized so as to preserve entailment

	(A ground sentence is entailed by new KB iff entailed by original KB)

	Idea: propositionalize KB and query, apply resolution, return result

	Problem: with function symbols, there are infinitely many ground terms,
		e.g., Father(Father(Father(John)))


	Theorem: Herbrand (1930). If a sentence α is entailed by an FOL KB, it is entailed by a finite subset of the propositionalized KB

	Idea: For n = 0 to ∞ do
		create a propositional KB by instantiating with depth-n terms
		see if α is entailed by this KB

	Problem: works if α is entailed, loops if α is not entailed

	Theorem: Turing (1936), Church (1936) Entailment for FOL is semi-decidable (algorithms exist that say yes to every entailed sentence, but no algorithm exists that also says no to every non-entailed sentence.)


	Propositionalization seems to generate lots of irrelevant sentences.

	E.g., from:
		"x King(x) Ù Greedy(x) Þ Evil(x)
		King(John)
		"y Greedy(y)
		Brother(Richard,John)

	it seems obvious that Evil(John), but propositionalization produces lots of facts such as Greedy(Richard) that are irrelevant

	With p k-ary predicates and n constants, there are p·n^k instantiations.


	We can get the inference immediately if we can find a substitution θ such that King(x) and Greedy(x) match King(John) and Greedy(y)

	θ = {x/John,y/John} works

	Unify(α,β) = θ if αθ = βθ
	p q θ
	Knows(John,x) Knows(John,Jane) {x/Jane}}
	Knows(John,x) Knows(y,OJ) {x/OJ,y/John}}
	Knows(John,x) Knows(y,Mother(y)) {y/John,x/Mother(John)}}
	Knows(John,x) Knows(x,OJ) {fail}

	Standardizing apart eliminates overlap of variables, e.g., Knows(z₁₇,OJ)


	To unify Knows(John,x) and Knows(y,z),
	θ = {y/John, x/z } or θ = {y/John, x/John, z/John}

	The first unifier is more general than the second.

	There is a single most general unifier (MGU) that is unique up to renaming of variables.
		MGU = { y/John, x/z }


	p₁', p₂', … , p_n', ( p₁ Ù p₂ Ù … Ù p_n Þq)
	qθ
	p₁' is King(John) p₁ is King(x)
	p₂' is Greedy(y) p₂is Greedy(x)
	θ is {x/John,y/John} q is Evil(x)
	q θ is Evil(John)

	GMP used with KB of definite clauses (exactly one positive literal)

	All variables assumed universally quantified