Deep Koopman Operators and the Identity Operator:

Let’s suppose that \(z_n = g(x_n) \in \mathbb{R}^n\) is an observable so we have:

\begin{equation} z_{n+1} = U \circ z_n \end{equation}

where \(U\) is the Koopman Operator.

We’ll find that \(U\) may be expressed in terms of the Identity Operator:

\begin{equation} U := I + \tilde{U} \end{equation}

where \(\tilde{U}\) is a linear operator. Thus, we have:

\begin{equation} z_{n+1} = U \circ z_n = (I + \tilde{U}) \circ z_n = z_n + \Delta z_n \end{equation}

and the iterated composition of \(U\) becomes:

\begin{equation} U^2 = (I + \tilde{U}) \circ (I + \tilde{U}) = I + 2 \tilde{U} + \tilde{U}^2 \end{equation}

This brief derivation suggests that skip connections(i.e. residual layers) are probably useful when approximating Koopman Operators with deep neural networks, as they significantly reduce the probability of information loss during neural network training.

Residual Networks and Euler’s method:

We may account for the Identity Operator in a neural network using residual layers \(F_{\theta}\):

\begin{equation} h_{t+1} = F_{\theta}(h_t) = h_t + f_{\theta}(h_t) = h_t + \Delta t \cdot \frac{f_{\theta}(h_t)}{\Delta t} \approx h_t + \Delta t \cdot f’_{\theta}(h_t) \end{equation}

which effectively allows us to implement the Euler method provided that \(f_{\theta} \approx \tilde{U}\).

In summary, this analysis indicates that Deep Koopman Operators with residual layers reduce information loss by simulating dynamical systems using the familiar one-step Euler Method. One of the implicit assumptions in this analysis is that approximating the identity operator isn’t necessarily easy for a neural network.

References:

  1. Bernard Koopman. Hamiltonian systems and Transformations in Hilbert Space.

  2. Steven L. Brunton. Notes on Koopman operator theory. 2019.

  3. Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems, pages 6571–6583, 2018.