Introduction:

The theory of Koopman operators has come a long way since Bernard Koopman’s seminal publication ‘Hamiltonian systems and Transformations in Hilbert Space’(1931), which I consider one of the greatest contributions to applied mathematics in the 20th Century. By generalising the mathematical apparatus of Quantum Mechanics to general dynamical systems, Koopman created a powerful method for the data-driven modelling of dynamical systems, a method which promises to revolutionise 21st Century science and engineering now that we have the computational infrastructure to realise the potential of this theory.

In this article, I introduce the theory of Koopman operators and explain how low-dimensional approximations of the Koopman operator capture the intrinsic geometry of dynamical systems by re-parametrizing the data in an intelligent way.

Dynamical systems and the Koopman operator:

The general theory of dynamical systems:

In the abstract, a dynamical system consists of a set of states and a rule for the evolution of a system state. This general viewpoint may be applied to practically any deterministic system that evolves in time.

Continuous-time dynamics may be represented as:

\begin{equation} \dot{x} = f(x) \end{equation}

where \(x \in S \subset \mathbb{R}^n\) and \(f:S \rightarrow \mathbb{R}^n\) is a vector field on that state space.

We may also consider dynamical systems given by the discrete-time map:

\begin{equation} \forall k \in \mathbb{Z}, x_{k+1} = T \circ x_k \end{equation}

where \(x_k \in S \subset \mathbb{R}^n\), \(k\) is a discrete-time index and \(T:S \rightarrow S\) is a dynamic map.

While the discrete-time representation of dynamical systems usually doesn’t show up in the representation of physical systems, we may use it to represent discrete-time sampling of these systems.

This representation is also more practical because the data collected from dynamical systems always comes in discrete-time samples.

Data as evaluations of functions of the state:

Within the context of dynamical systems, we may interpret data as knowledge of variables related to the system’s state. In fact, we may think of data as evaluations of functions of the state, known as observables.

The collection of observables forms a vector space:

Let \(g:S \rightarrow \mathbb{R}\) be a real-valued observable of the dynamical system. As a corollary of the Buckingham-Pi theorem, the collection of all such observables forms a vector space.

The Koopman operator \(U\) is a linear transform on this vector space given by:

\begin{equation} Ug(x) = g \circ T(x) \end{equation}

which in a discrete setting implies:

\begin{equation} Ug(x_k) = g \circ T(x_k) = g(x_{k+1}) \end{equation}

where the linearity of the Koopman operator follows from the linearity of the composition operator:

\begin{equation} U \circ (g_1 + g_2)(x) = g_1 \circ T(x) + g_2 \circ T(x) = Ug_1(x) + Ug_2(x) \end{equation}

So we may think of the Koopman operator as lifting dynamics of the state space to the space of observables.

Spectral analysis of the Koopman operator:

Koopman modes:

Let’s suppose \(\vec{x} \in M \subset \mathbb{R}^n\) contains all the information concerning the flow field at a particular time, so \(g(x)\) may be a vector of any quantity of interest such as velocity measurements.

If we let \(\phi_j(x): M \rightarrow \mathbb{R}\) denote eigenfunctions and \(\lambda_j \in \mathbb{C}\) denote eigenvalues of the Koopman operator, we have:

\begin{equation} \forall j \in \mathbb{N}, U\phi_j(x) = \lambda_j \cdot \phi_j(x) \end{equation}

If each of the components of \(\vec{g}\) lie within the span of the eigenfunctions \(\phi_j\) then we may expand the vector-valued \(\vec{g}\) in terms of the eigenfunctions:

\begin{equation} g(\vec{x}) = \sum_{j=1}^\infty \phi_j(x) \cdot \vec{v}_j \end{equation}

so we may think of \(\vec{g}\) as a linear combination of the eigenfunctions \(\phi_j\) of \(U\) where \(\vec{v}_j\) are the vector coefficients in the expansion.

By induction, we find the iterates of \(\vec{x}_0\) are given by:

\begin{equation} g(\vec{x_k}) = \sum_{j=1}^\infty U^k \circ \phi_j(\vec{x_0}) \cdot \vec{v_j} = \sum_{j=1}^\infty \lambda_j^k \cdot \phi_j(\vec{x_0}) \cdot \vec{v}_j \end{equation}

so we may think of the Koopman operator \(U\) as an infinite dimensional operator that acts on data from a Hilbert space.

The Koopman operator as a Discrete Fourier Transform for dynamical systems:

Building upon the notion that a Koopman operator may be decomposed into eigenfunctions we may show that in a large number of settings the Koopman operator performs an intelligent re-parametrization of the data by discovering regularities(ex. periodic behaviour). To be precise, the Koopman operator may be identified with a Discrete Fourier Transform for dynamical systems.

For concreteness, if we have a dataset of vectors sampled from a periodic system \(\forall k \in \mathbb{N}, x_{k+m}=x_k\):

\begin{equation} D = \{\vec{x_0}, …, \vec{x_{m-1}}\} \end{equation}

so we may compute its Discrete Fourier Transform:

\begin{equation} D \rightarrow DFT \rightarrow \hat{D} \end{equation}

\begin{equation} \forall j,k \in [0,m-1], x_k = \sum_{j=0}^{m-1} e^{\frac{2\pi ijk}{m}} \end{equation}

and so we may define a set of functions \(\phi_j:S \rightarrow \mathbb{C}\) by:

\begin{equation} \forall j,k \in [0,m-1],\phi_j(\vec{x_k}) = e^{\frac{2\pi ijk}{m}} \end{equation}

Then \(\phi_j\) are eigenfunctions of the Koopman operator \(U\) with eigenvalues \(e^{\frac{2\pi ijk}{m}}\) since:

\begin{equation} U\phi_j(x_k) = \phi_j \circ f(x_k) = \phi_j(x_{k+1}) = e^{\frac{2\pi ij(k+1)}{m}}= e^{\frac{2\pi ijk}{m}}\phi_j(x_k) \end{equation}

and therefore \(\text{rank}(U)=m\), so we may express the expansion as:

\begin{equation} \vec{x_k} = \sum_{j=0}^{m-1} \phi_j(x_k)\hat{x}_j \end{equation}

This result generalises to non-periodic systems when the dynamics are restricted to an attractor.

Numerical algorithms for Dynamic Mode Decomposition:

For real-world datasets, we may generalise the fourier decomposition proposed in the previous section using a data-driven re-parametrization known as Singular Value Decomposition. For concreteness, let’s suppose we have two time-shifted data matrices \(X_{n+1},X_n \in \mathbb{R}^{n \times m}\).

Given what we know about the Koopman operator \(U\), we may be inspired to find the linear transform \(A \in \mathbb{R}^{n \times n}\) such that:

\begin{equation} X_{n+1} = A \circ X_n \end{equation}

where we assume that given \(\vec{x} \in \mathbb{R}^n\):

\begin{equation} X_n = \{\vec{x}(t_k)\}_{k=1}^m \end{equation}

and one approach to computing \(A\) involves the pseudo-inverse of \(X\)(which minimises the Frobenius norm):

\begin{equation} A = \arg\min_{A} \lVert X_{n+1}-AX_n \rVert_F = X_{n+1} \cdot X_n^{+} \end{equation}

\begin{equation} X = U \Sigma V^* \implies X^+ = V \Sigma^+ U^* \end{equation}

where \(\Sigma^+ = (\Sigma^{-1})^{T}\).

In this brief presentation of DMD we have a relatively straightforward but powerful method that allows us to prioritise dominant eigenvalues and thereby capture the intrinsic geometry of the dynamical system. In some sense this algorithm also provides us with a constructive existence proof of the Manifold Hypothesis.

I could say more about DMD as there is an entire field of numerical algorithms devoted to the efficient approximation of Koopman operators but this is beyond the scope of this article.

Discussion:

What some readers might have realised by now is that Koopman operator theory provides us with a robust framework for approximating the data-generating process with a low-dimensional computer simulation. With advances in deep learning software, we may now automate the process at scale by approximating the eigenfunctions of the Koopman operator using deep neural networks. This in turn allows us to visualise low-dimensional embeddings and apply tools from topological data analysis to gain further insights into the low-dimensional topology of dynamical systems.

Moving forward, I am particularly excited about the application of Koopman operator approximations to modelling the evolution of cosmological phenomena and monitoring the behaviour of complex ecological systems. I also believe we may use representation theory to develop more powerful algorithms for approximating Koopman operators by exploiting invariants in a data stream.

References:

  1. Bernard Koopman. Hamiltonian systems and Transformations in Hilbert Space.

  2. Hassan Arbabi. Introduction to Koopman operator theory for dynamical systems. MIT. 2020.

  3. Steven L. Brunton. Notes on Koopman operator theory. 2019.

  4. Bethany Lusch et al. Deep learning for universal linear embeddings of non-linear dynamics. nature. 2018.

  5. C. Rowley et al. Spectral analysis of non-linear flows.

  6. Hassan Arbabi, Igor Mezić. Ergodic theory, Dynamic Mode Decomposition and Computation of Spectral properties of the Koopman operator. 2017.

  7. Anastasiya Salova et al. Koopman operator and its approximation for systems with symmetries. 2019.

  8. Aidan Rocke. Physical interpretation of the Manifold Hypothesis. MathOverflow. 2020.