Constructing Markov Logic Networks from First-Order Default Rules

Abstract

Expert knowledge can often be represented using default rules of the form “if A then typically B”. In a probabilistic framework, such default rules can be seen as constraints on what should be derivable by MAP-inference. We exploit this idea for constructing a Markov logic network $\mathcal {M}$ from a set of first-order default rules D, such that MAP inference from $\mathcal {M}$ exactly corresponds to default reasoning from D, where we view first-order default rules as templates for the construction of propositional default rules. In particular, to construct appropriate Markov logic networks, we lift three standard methods for default reasoning. The resulting Markov logic networks could then be refined based on available training data. Our method thus offers a convenient way of using expert knowledge for constraining or guiding the process of learning Markov logic networks.

A Proofs

Here we provide formal justifications for the transformations presented in this paper⁹. We start by proving correctness of the ground transformations.

Proposition 1

Let $\varDelta$ be a set of default rules. Let $\varDelta ^{rat}$, $\varDelta ^{lex}$ and $\varDelta ^{ent}$ be the rational, lexicographic and maximum entropy closure of $\varDelta$, respectively. Let $\mathcal {M}^{rat}$, $\mathcal {M}^{lex}$ and $\mathcal {M}^{ent}$ be Markov logic networks obtained from $\varDelta$ by Transformations 1, 2 and 3, respectively. Then the following holds for any default rule $\alpha {\,\mid \!\sim \,}\beta$:

• 1. $\alpha {\,\mid \!\sim \,}\beta \in \varDelta ^{rat}$ if and only if $(\mathcal {M}^{rat}, \{ \alpha \}) \vdash _{\textit{MAP}}\beta$,

• 2. $\alpha {\,\mid \!\sim \,}\beta \in \varDelta ^{lex}$ if and only if $(\mathcal {M}^{lex}, \{ \alpha \}) \vdash _{\textit{MAP}}\beta$,

• 3. $\alpha {\,\mid \!\sim \,}\beta \in \varDelta ^{ent}$ if and only if $(\mathcal {M}^{ent}, \{ \alpha \}) \vdash _{\textit{MAP}}\beta$.

Proof

Throughout the proof, let $\varDelta = \varDelta _1 \cup \varDelta _2 \cup \dots \cup \varDelta _k$ be the Z-ordering of $\varDelta$.

1. Let $\alpha {\,\mid \!\sim \,}\beta \in \varDelta ^{rat}$ be a default rule. Let j be the smallest index such that $\varDelta _{\alpha }^{rat} = \{\lnot \gamma \vee \delta | \gamma {\,\mid \!\sim \,}\delta \in \varDelta _j \cup ... \cup \varDelta _k \} \cup \{ \alpha \}$ is consistent. Recall that $\alpha {\,\mid \!\sim \,}\beta \in \varDelta ^{rat}$ if and only if $\varDelta _\alpha ^{rat} \models \beta$. By the construction of the MLN $\mathcal {M}^{rat}$ it must hold that $(\mathcal {M}^{rat},\{ \alpha \}) \vdash _{\textit{MAP}}\lnot a_i$ for every $i < j$ and also $(\mathcal {M}^{rat},\{ \alpha \}) \vdash _{\textit{MAP}}a_i$ for all $i \ge j$. Therefore all $\lnot \alpha \vee \beta$, such that $\alpha {\,\mid \!\sim \,}\beta \in \varDelta _i$ where $i \ge j$, must be true in all most probable worlds of $(\mathcal {M}^{rat},\{\alpha \})$. But then necessarily we have: if $\varDelta _\alpha ^{rat} \models \beta$ then $(\mathcal {M}^{rat},\{ \alpha \}) \vdash _{\textit{MAP}}\beta$. Similarly, to show the other direction of the implication, let us assume that $(\mathcal {M}^{rat},\{ \alpha \}) \vdash _{\textit{MAP}}\beta$. Then we can show using basically the identical reasoning as for the other direction that the set of formulas $\lnot \alpha \vee \beta$ which must be satisfied in all most probable worlds of $(\mathcal {M}^{rat},\{\alpha \})$ is equivalent to the set of formulas in $\varDelta _\alpha ^{rat}$.

2. It holds that $\alpha {\,\mid \!\sim \,}\beta \in \varDelta ^{lex}$ if and only if $\beta$ is true in all lex-preferred models of $\alpha$, i.e. $\forall \omega \in \llbracket \alpha \rrbracket : (\omega \not \models \beta ) \Rightarrow \exists \omega '\in \llbracket \alpha \rrbracket : \omega ' \prec \omega$ where $\prec$ is the lex-preference relation based on Z-ordering defined in Sect. 2.2. What we need to show is that for any possible worlds $\omega$, $\omega '$ it holds $\omega \prec \omega '$ if and only $P_{\mathcal {M}^{lex}}(\omega ) > P_{\mathcal {M}^{lex}}(\omega ')$ where $P_{\mathcal {M}^{lex}}$ is the probability given by the MLN $\mathcal {M}^{lex}$, from which correctness of the lexicographic transformation will follow. But this actually follows immediately from the way we set the weights in Transformation 2, as the penalty for not satisfying one formula corresponding to a default rule in $\varDelta _i$ is greater than the sum of penalties for not satisfying all formulas corresponding to the default rules in $\bigcup _{j < i} \varDelta _j$.

3. This follows directly from the results in the paper [10] in which maximum entropy closure was introduced. An explicit ranking function on possible worlds was derived in that paper, which we explicitly use in the transformation.

Next we show correctness of the non-ground transformations. We start by proving properties of the non-ground counterpart of Z-ordering.

Proposition 2

Let $\varDelta ^*$ be a default theory and $\mathcal {C}$ be a set of constants (universe). Let $\varDelta$ be the set of groundings¹⁰ of default rules from $\varDelta ^*$. Let $\varDelta = \varDelta _1 \cup \dots \cup \varDelta _k$ be Z-ordering of the set of ground default rules $\varDelta$. Let $\varDelta _1^* \cup \dots \cup \varDelta _k^*$ be as defined by Eq. 3. Then a ground default rule $\alpha {\,\mid \!\sim \,}\beta$ is in $\varDelta _i$ if and only if a rule isomorphic to $\textit{variabilize}(\alpha {\,\mid \!\sim \,}\beta )$ is in $\varDelta _i^*$.

Proof

(Sketch) This proposition follows from the simple observation that Eq. 3 is equivalent to checking whether the ground default rule $\alpha {\,\mid \!\sim \,}\beta$ is tolerated by the set of groundings of the default rules $\gamma {\,\mid \!\sim \,}\delta \in \varDelta ^* \setminus (\varDelta ^*_1 \cup \dots \cup \varDelta ^*_{j-1})$ (because we explicitly ask there about existence of a Herbrand model¹¹ with universe $\mathcal {C}$). Since the answer, whether it is tolerated or not, must be the same for every default rule weakly isomorphic to $\alpha {\,\mid \!\sim \,}\beta$, it follows that this is equivalent to checking this condition for all groundings of $\textit{variabilize}(\alpha {\,\mid \!\sim \,}\beta )$, which must then necessarily give us an equivalent result to what we would obtain by Z-ordering performed on the explicitly enumerated groundings. The statement of the proposition then follows from this.

In other words, what the above proposition states, is that if we replace non-ground rules in the particular $\varDelta _i^*$’s by all their groundings then this partitioning of ground default rules must be equivalent to what we would obtain by directly Z-ordering the ground default rules in the set $\mathcal {R}$.

Proposition 3

Let $\varDelta ^*$ be a set of non-ground default rules and $\mathcal {C}$ be a set of constants (universe). Let $\varDelta ^{rat}$, $\varDelta ^{lex}$ and $\varDelta ^{ent}$ be the rational, lexicographic and maximum entropy closure, respectively, of the set of default rules obtained by grounding $\varDelta ^*$. Let $\mathcal {M}^{rat}$, $\mathcal {M}^{lex}$ and $\mathcal {M}^{ent}$ be Markov logic networks obtained from $\varDelta ^*$ by Transformations 4, 5 and 6, respectively. Then the following holds for any ground default rule $\alpha {\,\mid \!\sim \,}\beta$:

• 1. $\alpha {\,\mid \!\sim \,}\beta \in \varDelta ^{rat}$ if and only if $(\mathcal {M}^{rat}, \{ \alpha \}) \vdash _{\textit{MAP}}\beta$,

• 2. $\alpha {\,\mid \!\sim \,}\beta \in \varDelta ^{lex}$ if and only if $(\mathcal {M}^{lex}, \{ \alpha \}) \vdash _{\textit{MAP}}\beta$,

• 3. $\alpha {\,\mid \!\sim \,}\beta \in \varDelta ^{ent}$ if and only if $(\mathcal {M}^{ent}, \{ \alpha \}) \vdash _{\textit{MAP}}\beta$.

Proof

1. This case follows from Propositions 1 and 2 by noticing that the constructed MLNs, when grounded, are the same as what the ground Transformation 1 would produce if applied on all groundings of default rules from $\varDelta ^*$.

2. From Proposition 2 we have that, if we ground the MLN produced by Transformation 5, then the structure of the MLN will be identical to what we would obtain if we applied Transformation 2 on all groundings of default rules from $\varDelta ^*$. While the weights of the formulas are not the same, it is still guaranteed that $\omega \prec \omega '$ if and only $P_{\mathcal {M}^{lex}}(\omega ) > P_{\mathcal {M}^{lex}}(\omega ')$, where $\prec$ is the lex-preference relation. This is because the term $|\mathcal {C}|^{|vars(\alpha {\,\mid \!\sim \,}\beta )|}$, which is used to define the weights in Transformation 5, is an upper bound on the number of groundings of a default rule $\alpha {\,\mid \!\sim \,}\beta$ (this implies that the sum of all weights of groundings of formulas in the MLN which correspond to default rules from $\bigcup _{i < j} \varDelta _i$ will be smaller than the weight of a single formula corresponding to a default rule from $\varDelta _j$ which is what we need).

3. (Sketch) To show the last part of this proposition, we would basically need to replicate a more detailed reasoning from the proof of Proposition 2 because maximum entropy closure needs to create a partitioning of the set of default rules which refines Z-ordering. Since no new ideas are needed for this proof and because of space limitations, we omit details. The basic idea is the same as for the non-ground Z-ordering – we only process representatives of the non-ground default rules and we can show that we would obtain an equivalent result if we processed all groundings of the default rules by the procedure from Transformation 3.