Fuzzy World: A Tool Training Agent from Concept Cognitive to Logic Inference

Abstract

Not like many visual systems or NLP frameworks, human generally use both visual and semantic information for reasoning tasks. In this paper, we present a 3D virtual simulation learning environment Fuzzy World based on gradual learning paradigm to train visual-semantic reasoning agent for complex logic reasoning tasks. Furthermore our baseline approach employed semantic graphs and deep reinforcement learning architecture shows the significant performance over the tasks.

Appendix

1.1 6.1 Using Second Order Derivative Gradient for Cross Training Parameter

Notice that the prediction of the model $\mathcal {P}(L_A|V, L_Q)$ is in one-hot form of space concept like:[up and down, left and right, top left and bottom right...], then the loss of last layer employed softmax cross entropy loss is $\mathcal {L}(A_S) = -\hat{y} \odot log(f_{softmax}(A_S \odot C^{*T}))$. The next is the provement of an upper bound of $\hat{\mathcal {L}}(A+ \alpha \varDelta A)$.

Note that the updating of parameters A takes the simple SGD: $A^{t+1} \leftarrow A^t+\alpha \nabla _A \hat{\mathcal {L}}$.

Theorem 1

When $\nabla ^2_A \hat{\mathcal {L}} \le MI$, we have$\hat{\mathcal {L}}(A+ \alpha \varDelta A) \le \hat{\mathcal {L}}(A) + \gamma ||\nabla _A \hat{\mathcal {L}}||^2$.

Proof

Easy to know $-\nabla _A \hat{\mathcal {L}}(A) = \varDelta A$, do Taylor expansion to $\hat{\mathcal {L}}(A+ \alpha \varDelta A)$:

$$\hat{\mathcal {L}}(A+ \alpha \varDelta A) = \hat{\mathcal {L}}(A)+ \alpha \nabla _A \hat{\mathcal {L}}(A) \odot \varDelta A +\nabla ^2_A \hat{\mathcal {L}} ||\varDelta A||^2 \alpha ^2 /2 $$

$$\le \hat{\mathcal {L}}(A)+ \alpha \nabla _A \hat{\mathcal {L}} \odot (-\nabla _A \hat{\mathcal {L}}) +M||\varDelta A||^2 \alpha ^2 /2 $$

$$= \hat{\mathcal {L}}(A)+(\alpha ^2M /2 - \alpha )||\nabla _A \hat{\mathcal {L}}||^2 $$

Now let $\gamma = \alpha ^2M /2 - \alpha \le 0$ then the below is satisfied:

$$\hat{\mathcal {L}}(A+ \alpha \varDelta A) \le \hat{\mathcal {L}}(A+ \alpha \varDelta A) +(\alpha -\alpha ^2M /2 )||\nabla _A \hat{\mathcal {L}}||^2 \le \hat{\mathcal {L}}(A)$$

.

Fig. 5.

The Three-level Learning Paradigm: task examples from object classification to concept cognitive and logic reasoning, semantic graphs are generated and used in logic reasoning tasks.