Recurrent Neural Networks: Some Systems-Theoretic Aspects

Abstract

Recurrent nets have been introduced in control, computation, signal processing, optimization, and associate memory applications. Given matrices A ∈ ℝ^{n ×n}, B ∈ ℝ^{n ×m}, C ∈ ℝ^{p ×n}, as well as a fixed Lipschitz scalar function σ : ℝ → ℝ, the continuous time recurrent network Σ with activation function σ and weight matrices (A, B,C) is given by:

$$\frac{{dx}}{{dt}}(t) = {\overrightarrow \sigma ^{(n)}}\left( {Ax(t) + Bu(t)} \right) , y\left( t \right) = Cx\left( t \right)$$

(1)

where ${\overrightarrow \sigma ^{\left( n \right)}}$: ℝⁿ → ℝⁿ is the diagonal map

$${\overrightarrow \sigma ^{\left( n \right)}}:\left( {\begin{array}{*{20}{c}}{{x_1}} \\ \vdots \\ {{x_n}}\end{array}} \right) \mapsto \left({\begin{array}{*{20}{c}}{\sigma \left( {{x_1}} \right)} \\ \vdots \\ {\sigma \left( {{x_n}} \right)} \end{array}} \right)$$

(2)