Motivation:

Let’s suppose we have a smooth function \(g: \mathbb{R} \rightarrow \mathbb{R}\) that describes recently-discovered physical phenomena whose behaviour is non-linear in the neighbourhood of \(x=a\). In order to describe the behaviour of \(g\) near \(x=a\), we may define the function space \(F\) of Taylor polynomials:

\begin{equation} \lim_{x \to a} \frac{g(x)}{f(x)} = 1 \implies f \in F \end{equation}

where \(F\) contains an Occam approximation of \(g\) in the neighbourhood of \(x=a\) if it satisfies two properties:

\(1.\) Non-linearity in the neighbourhood of \(x=a\) implies that the lowest-order Taylor polynomial in \(F\) must be greater than or equal to two. This requirement implies that \(F\) contains descriptions of the behaviour of \(g\).

\(2.\) \(F\) is constrained to satisfy the prefix property. Intuitively, this means that \(f' \in F\) if and only if it appears as a prefix of some \(f \in F\) where \(f \neq f'\). A more precise definition of this prefix property shall be given in the next paragraph in terms of computation graphs.

Computation graphs of Taylor polynomials:

In order to define Occam’s approximation of \(g\) near \(x=a\), we may represent each polynomial \(f \in F\) in terms of its computation graph \(G \circ f\):

\begin{equation} f = \sum_{n=1}^N a_n(x) \implies G \circ f = \{a_n \}_{n=1}^N \end{equation}

where \(\forall n \in [1,N], a_n(x) = \alpha_n \cdot x^{n-1}\) for some constant \(\alpha_n \in \mathbb{R}\). Given \(G \circ f\), we may also readily recover \(f\):

\begin{equation} f = G^{-1} \circ (G \circ f) \end{equation}

Using (2), we may represent the computation of a Taylor polynomial \(f\) by a directed Hamiltonian path from the node \(a_1\) to the node \(a_n\) where the edge between consecutive nodes is the addition operation. In this formalism, \(f\) is a sub-computation of \(f' \in F\) if \(G \circ f\) appears as a sub-graph of \(G \circ f'\):

\begin{equation} G \circ f \subset G \circ f’ \end{equation}

In fact, (4) captures the prefix property of \(F\) in the sense that:

\begin{equation} f’ \in F \implies \exists f \in F, f’ \neq f \land G \circ f \subset G \circ f’ \end{equation}

and given (5), Occam’s approximation of \(g\) near \(x=a\) is none other than:

\begin{equation} f = G^{-1} \circ \bigcap_{f’ \in F} G \circ f’ \end{equation}

and for the sake of brevity:

\begin{equation} f = \mathcal{O}\big(\lim_{x \to a} g(x)\big) \end{equation}

The case where the lowest-order Taylor polynomial is infinite:

If the lowest-order Taylor polynomial in \(F\) is infinite, we may use a space of special functions \(H\) instead in order to guarantee that all computation graphs have a finite number of nodes.

By requiring that \(H\) satisfy the prefix property, we may guarantee that Occam’s approximation of a special function \(g\) near \(x=a\) with respect to \(H\) is well-defined.

Discussion:

Due to the scope of such a general theory, I think it is normal to expect that such a theory would go through several iterations in order to iron out difficulties which arise in its application. However, what I have presented here is the first iteration of such a theory.

References:

  1. Nikiforov, A. F. and Uvarov, V. B. Special Functions of Mathematical Physics: A Unified Introduction with Applications. Boston, MA: Birkhäuser, 1988.

  2. Evgeny Lifshitz and Lev Landau. Statistical Physics. Butterworth-Heinemann. 1980.