<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Lekan Molu</title>
    <description>Geeks Shall Inherit the Earth
</description>
    <link>http://scriptedonachip.com/</link>
    <atom:link href="http://scriptedonachip.com/atom.xml" rel="self" type="application/rss+xml" />
    <pubDate>Sat, 14 Nov 2026 21:29:32 +0000</pubDate>
    <lastBuildDate>Sat, 14 Nov 2026 21:29:32 +0000</lastBuildDate>
    <generator>Jekyll v3.10.0</generator>
    
      <item>
        <title>HJ-Gauss: Sampling-based HJ Reachability</title>
        <description>&lt;h1 id=&quot;sampling-your-way-out-of-the-curse-of-dimensionality&quot;&gt;Sampling Your Way Out of the Curse of Dimensionality&lt;/h1&gt;

&lt;script type=&quot;text/x-mathjax-config&quot;&gt;
MathJax.Hub.Config({
  TeX: { equationNumbers: { autoNumber: &quot;AMS&quot; } }
});
&lt;/script&gt;

&lt;blockquote&gt;
  &lt;p&gt;This post accompanies the current 28-page paper &lt;strong&gt;&lt;a href=&quot;/downloads/Papers/hjgauss.pdf&quot;&gt;Sampling-based HJ Reachability&lt;/a&gt;&lt;/strong&gt;.
The paper’s authoritative formulation retains the complete transformation residual. The validated implementation is in the
&lt;a href=&quot;https://github.com/robotsorcerer/levelsetpy/tree/main/levelsetpy/monte_carlo&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;&lt;strong&gt;monte_carlo&lt;/strong&gt;&lt;/a&gt; package of &lt;a href=&quot;https://github.com/robotsorcerer/levelsetpy&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;&lt;strong&gt;levelsetpy&lt;/strong&gt;&lt;/a&gt;.
Animations: &lt;a href=&quot;/downloads/Papers/assets/anim/roysoc_rockets.gif&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Rockets&lt;/a&gt; · &lt;a href=&quot;/downloads/Papers/assets/anim/roysoc_dubins.gif&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Dubins vehicles&lt;/a&gt; · &lt;a href=&quot;/downloads/Papers/assets/anim/roysoc_double_integrator.gif&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Double integrator&lt;/a&gt; · &lt;a href=&quot;/downloads/Papers/assets/anim/roysoc_murmuration.gif&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Starling model&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The &lt;strong&gt;&lt;em&gt;reliability&lt;/em&gt;&lt;/strong&gt;&lt;sup id=&quot;fnref:reliability&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:reliability&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;1&lt;/a&gt;&lt;/sup&gt; of the modern automation algorithms that we design (e.g., in AI, modern cyberphysical systems etc.) has become paramount given the dangers that may occur if nominally envisioned system performance falters. The need for scalable and faster numerical algorithms in software for the reliable &lt;strong&gt;&lt;em&gt;verification&lt;/em&gt;&lt;/strong&gt;&lt;sup id=&quot;fnref:verification&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:verification&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;2&lt;/a&gt;&lt;/sup&gt;
and &lt;strong&gt;&lt;em&gt;validation&lt;/em&gt;&lt;/strong&gt;&lt;sup id=&quot;fnref:validation&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:validation&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;3&lt;/a&gt;&lt;/sup&gt;
has become timely given the emergence of complexity of many contemporary (complex) systems. For the rest of this blog post safety shall be taken to mean verification.&lt;/p&gt;

&lt;p&gt;Backward reachable tubes (BRTs) are a verification means of certifying that a system’s trajectory cannot be forced into an unsafe region no matter what a disturbance throws at it. Computing one amounts to solving a Hamilton-Jacobi-Isaacs (HJI)
PDE and isolating the zero sublevel set of the resulting value function. Since there is no 
is computational, not conceptual: every classical solver — level-set methods, essentially
non-oscillatory finite differences, anything built around a mesh — discretizes the state
space on a grid with \(M\) points per dimension, and the memory bill is \(O(M^n)\).
Push past five or six state dimensions and no amount of GPU acceleration saves you; the
grid itself no longer fits anywhere.&lt;/p&gt;

&lt;p&gt;HJ-Gauss replaces the grid with samples. The value function becomes a Gaussian
expectation, evaluated Monte Carlo style at whichever states you actually care about.
Memory becomes \(O(N \cdot n)\) — linear in the state dimension \(n\), not exponential —
and the paper backs this up with a 45-dimensional, 15-rocket pursuit-evasion game and a
100,000-bird starling murmuration, neither of which any grid solver can represent at all.&lt;/p&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;1-why-this-is-hard-the-omn-wall&quot;&gt;&lt;strong&gt;1. Why this is hard: the \(O(M^n)\) wall&lt;/strong&gt;&lt;/h3&gt;

&lt;p&gt;A central concern in trustworthy machine learning is certifying that a learned
controller, neural policy, or planner will not violate safety constraints at
deployment — safe reinforcement learning, model-based RL with learned dynamics, any
pipeline where a model must act in a continuous, possibly high-dimensional space under
adversarial or uncertain inputs. Computing a BRT means solving&lt;/p&gt;

\[\begin{align}
v_t + H(t, x, Dv) &amp;amp;= 0 \quad \text{in } \Omega \times (0,T], \label{eq:hji} \\
v(0,x) &amp;amp;= g(0,x) \quad \text{on } \Omega \times \{t=0\}, \label{eq:hji-terminal}
\end{align}\]

&lt;p&gt;where every symbol is worth naming explicitly, since the paper’s own reviewers rightly
would not let this slide:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;\(x \in \Omega\) is the &lt;strong&gt;state&lt;/strong&gt;, belonging to the state-space domain
\(\Omega \subseteq \mathbb{R}^n\) (e.g. relative position and heading for a
pursuit-evasion game);&lt;/li&gt;
  &lt;li&gt;\(\dot x = f(t,x,u,w)\) is the &lt;strong&gt;system dynamics&lt;/strong&gt;, with control input \(u\) (the
agent trying to stay safe) and disturbance/adversary input \(w\) (the agent, or worst
case, trying to force capture);&lt;/li&gt;
  &lt;li&gt;\(v(t,x)\) is the &lt;strong&gt;value function&lt;/strong&gt; and \(Dv\) its spatial gradient — the vector of
partial derivatives of \(v\) with respect to \(x\);&lt;/li&gt;
  &lt;li&gt;\(p\) is the &lt;strong&gt;co-state&lt;/strong&gt;, a placeholder variable for “whatever gradient \(Dv\) gets
plugged in” — \(H(t,x,p)\) is defined independently of \(v\), and \(H(t,x,Dv)\) is
that same function evaluated at \(p = Dv(t,x)\);&lt;/li&gt;
  &lt;li&gt;\(H(t,x,p) = \max_u \min_w \langle p, f(t,x,u,w)\rangle\) is the &lt;strong&gt;Hamiltonian&lt;/strong&gt;: the
worst case (over disturbance \(w\)), best case (over control \(u\)) rate of change of
the value along the dynamics, in the co-state direction \(p\);&lt;/li&gt;
  &lt;li&gt;\(g(0,x)\) is the &lt;strong&gt;terminal/initial datum&lt;/strong&gt; — a cost function encoding the target set
to be reached or avoided (in practice, a signed distance to that set);&lt;/li&gt;
  &lt;li&gt;\(M\) is grid points per axis, \(n\) the state dimension.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The BRT is the zero sublevel set \(\{x : v(0,x) \le 0\}\): states from which, no matter
what \(w\) does, \(u\) can force the system into (or keep it out of) the target set
within the horizon \(T\). Grid-based solvers discretize \(\Omega\) with \(M\) points per
axis and compute the spatial gradient \(Dv\) at each grid node with &lt;strong&gt;ENO&lt;/strong&gt; (Essentially
Non-Oscillatory) or &lt;strong&gt;WENO&lt;/strong&gt; (Weighted ENO) finite-difference stencils: rather than a
single fixed upwind stencil, ENO adaptively selects the &lt;em&gt;smoothest&lt;/em&gt; of several candidate
stencils at each node, and WENO instead takes a &lt;em&gt;weighted combination&lt;/em&gt; of all of them,
with weights that favor smooth candidates and suppress oscillatory ones — both are
built to avoid the spurious ringing a naive centered/upwind scheme produces near the
kinks and shocks that value functions of HJ PDEs generically have, e.g. at the BRT
boundary itself. For a full treatment of these schemes as implemented for reachability,
see &lt;a href=&quot;https://doi.org/10.1109/CDC56724.2024.10886640&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;Molu, “The Python LevelSet Toolbox (LevelSetPy),” IEEE CDC 2024&lt;/a&gt;
and Molu, “LevelSetPy: A GPU-Accelerated Package for Hyperbolic Hamilton-Jacobi Partial
Differential Equations’ Solubility,” &lt;em&gt;ACM Transactions on Mathematical Software&lt;/em&gt;, 2025 —
the grid-based reference solver used for every comparison in this post. A 6-dimensional
problem at \(M=100\) points per axis already needs \(10^{12}\) grid cells regardless of
which finite-difference scheme computes \(Dv\) at each one — computationally
inaccessible, and it only gets worse from there.&lt;/p&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;2-the-core-idea-cole-hopf-turns-the-pde-into-an-expectation&quot;&gt;&lt;strong&gt;2. The core idea: Cole-Hopf turns the PDE into an expectation&lt;/strong&gt;&lt;/h3&gt;

&lt;p&gt;Fix a viscosity parameter \(\delta &amp;gt; 0\) and consider the &lt;strong&gt;viscous&lt;/strong&gt; HJ equation
(Crandall-Lions vanishing-viscosity regularization):&lt;/p&gt;

\[\begin{align}
v^\delta_t + H(t,x,Dv^\delta) = \frac{\delta}{2}\Delta v^\delta. \label{eq:viscous-hj}
\end{align}\]

&lt;p&gt;When the Hamiltonian is exactly quadratic in the co-state, \(H = \tfrac{1}{2}|p|^2\), the
substitution \(\omega^\delta := \exp(-v^\delta/\delta)\) is the classical &lt;strong&gt;Cole-Hopf
transform&lt;/strong&gt;: \(\omega^\delta\) satisfies the plain heat equation
\(\omega^\delta_t = (\delta/2)\Delta\omega^\delta\) with no residual whatsoever — the full
derivation, worked line by line, is in &lt;strong&gt;&lt;a href=&quot;#appendix-a-the-cole-hopf-reduction-in-full&quot;&gt;Appendix A&lt;/a&gt;&lt;/strong&gt;
at the bottom of this post. A heat equation has an explicit Green’s-function solution, and
by the Feynman-Kac formula that convolution is exactly a Gaussian expectation:&lt;/p&gt;

\[\begin{align}
v^\delta(t,x) = -\frac{1}{c}\log\; \mathbb{E}_{y \sim \mathcal{N}(x,\, \delta t I_n)}
\Big[\exp\big(-c\, g(y)\big)\Big], \qquad c = \frac{1}{\delta}. \label{eq:value-expectation}
\end{align}\]

&lt;p&gt;That’s the entire trick. The value function at any point \(x\) is a log-sum-exp over
Gaussian draws centered at \(x\) — no grid, no discretization, just samples. The spatial
gradient has an equally closed form (a self-normalized importance-weighted mean):&lt;/p&gt;

\[\begin{align}
Dv^\delta(t,x) = \frac{1}{t\,\delta\,c}\left(x -
\frac{\mathbb{E}_{y}\big[y \cdot \exp(-c\,g(y))\big]}{\mathbb{E}_{y}\big[\exp(-c\,g(y))\big]}\right). \label{eq:gradient-expectation}
\end{align}\]

&lt;hr /&gt;

&lt;h3 id=&quot;3-general-hamiltonians-quasi-linearization-by-picard-iteration&quot;&gt;&lt;strong&gt;3. General Hamiltonians: quasi-linearization by Picard iteration&lt;/strong&gt;&lt;/h3&gt;

&lt;p&gt;Real reachability problems don’t have quadratic Hamiltonians — pursuit-evasion games,
Dubins vehicles, and multi-agent systems all have genuinely nonlinear, nonconvex \(H\).
For a general \(H\), define the &lt;strong&gt;spatially-varying coefficient&lt;/strong&gt;&lt;/p&gt;

\[\begin{align}
c(t,x) = \frac{2}{\delta}\cdot \frac{H(t,x,Dv^\delta)}{|Dv^\delta|^2}. \label{eq:frozen-coeff}
\end{align}\]

&lt;p&gt;Substituting \(\omega^\delta = \exp(-c\,v^\delta)\) into the viscous PDE no longer gives a
clean heat equation — it produces a heat equation &lt;em&gt;plus a residual&lt;/em&gt;, split into an
algebraic part (annihilated exactly by the choice of \(c\) above) and a &lt;strong&gt;derivative&lt;/strong&gt;
part built from \(c_t\), \(Dc\), \(\Delta c\) that no algebraic choice of \(c\) can
remove. The scheme is therefore a &lt;strong&gt;Picard quasi-linearization&lt;/strong&gt;: freeze \(c^{(k)}\) at
the current iterate, solve the resulting &lt;em&gt;locally-quadratic&lt;/em&gt; surrogate problem exactly
via the closed-form expectation above, recover \(v^{(k+1)}\) and \(Dv^{(k+1)}\), update
\(c^{(k+1)}\), repeat.&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Algorithm 1 — Quasi-Linearization via Cole-Hopf
  Fix ε &amp;gt; 0.  v⁽⁰⁾(x) = g(x),  c⁽⁰⁾ = 2·H(x, Dg) / (δ·|Dg|²)
  for k = 0, 1, 2, … do
      freeze c⁽ᵏ⁾ at the current iterate
      solve the heat equation exactly:
          v⁽ᵏ⁺¹⁾(x) = -(1/c⁽ᵏ⁾)·log 𝔼_y~N(x,δtI)[ exp(-c⁽ᵏ⁾ · g(y)) ]
      recover Dv⁽ᵏ⁺¹⁾ (closed-form importance-weighted mean)
      update c⁽ᵏ⁺¹⁾ = 2·H(x, Dv⁽ᵏ⁺¹⁾) / (δ·|Dv⁽ᵏ⁺¹⁾|²)
      if ‖v⁽ᵏ⁺¹⁾ - v⁽ᵏ⁾‖ / ‖v⁽ᵏ⁾‖ &amp;lt; ε: break
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The iteration converges to a &lt;em&gt;fixed point of the surrogate map&lt;/em&gt; — that’s a genuinely
different object from the true viscous solution in general, and the gap between the two
(the “quasi-linearization defect”) is bounded in the paper rather than assumed away. The
scheme is exact precisely when \(H\) is quadratic; away from that, the defect is
controlled by how fast \(c\) varies spatially — smallest near smooth regions of the value
function, largest near the reachability boundary where the co-state direction changes
sharply.&lt;/p&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;4-the-actual-code&quot;&gt;&lt;strong&gt;4. The actual code&lt;/strong&gt;&lt;/h3&gt;

&lt;p&gt;The whole solver is one object. From
&lt;a href=&quot;https://github.com/robotsorcerer/levelsetpy/blob/main/monte_carlo/src/hj_sampler.py&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;&lt;code&gt;src/hj_sampler.py&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;kn&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;src.config&lt;/span&gt; &lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;SolverConfig&lt;/span&gt;
&lt;span class=&quot;kn&quot;&gt;from&lt;/span&gt; &lt;span class=&quot;nn&quot;&gt;src.hj_sampler&lt;/span&gt; &lt;span class=&quot;kn&quot;&gt;import&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;HJReachabilitySampler&lt;/span&gt;

&lt;span class=&quot;n&quot;&gt;cfg&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;SolverConfig&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;delta&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mf&quot;&gt;0.08&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;           &lt;span class=&quot;c1&quot;&gt;# viscosity; approximation error ~ O(sqrt(delta))
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;num_samples&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;14_000&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;   &lt;span class=&quot;c1&quot;&gt;# Monte Carlo samples per evaluated point
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;max_quasi_iters&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;20&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;   &lt;span class=&quot;c1&quot;&gt;# Picard quasi-linearization iterations
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;quasi_tol&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mf&quot;&gt;1e-5&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;t_start&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;t_end&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mf&quot;&gt;0.5&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;gradient_mode&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;b17&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;  &lt;span class=&quot;c1&quot;&gt;# importance-weighted gradient estimator
&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;sampler&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;HJReachabilitySampler&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;hamiltonian&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;terminal_cost&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;cfg&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;v&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;history&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sampler&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;solve_quasi_linear&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;eval_points&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;t&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mf&quot;&gt;0.0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;  &lt;span class=&quot;c1&quot;&gt;# eval_points: (M, n)
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;eval_points&lt;/code&gt; can be a uniform grid for visualization, or scattered arbitrarily — the
&lt;em&gt;solve&lt;/em&gt; never discretizes anything, so nothing about the algorithm cares how you chose
those \(M\) query points. Under the hood, the per-point Gaussian expectation
(&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;src/heat_solver.py&lt;/code&gt;) is a stabilized log-sum-exp:&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;mc_value_at_point&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;key&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;x&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;t&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;T&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;delta&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;c&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;terminal_cost_fn&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;num_samples&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;):&lt;/span&gt;
    &lt;span class=&quot;s&quot;&gt;&quot;&quot;&quot;v(t, x) via a Monte Carlo Gaussian expectation, log-sum-exp stabilized.&quot;&quot;&quot;&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;n&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;x&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;shape&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;jnp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;sqrt&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;jnp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;maximum&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;delta&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;T&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;t&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;),&lt;/span&gt; &lt;span class=&quot;mf&quot;&gt;1e-30&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;

    &lt;span class=&quot;n&quot;&gt;z&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;jax&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;random&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;normal&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;key&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;shape&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;num_samples&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;n&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;y&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;x&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;None&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;:]&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sigma&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;z&lt;/span&gt;                      &lt;span class=&quot;c1&quot;&gt;# (N, n) Gaussian draws
&lt;/span&gt;    &lt;span class=&quot;n&quot;&gt;g_vals&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;vmap&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;terminal_cost_fn&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;y&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;               &lt;span class=&quot;c1&quot;&gt;# (N,)
&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;exponents&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;c&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;g_vals&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;max_exp&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;jnp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;max&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;exponents&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;log_mean_exp&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;max_exp&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;jnp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;log&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;jnp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;mean&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;jnp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;exp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;exponents&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;max_exp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)))&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;mf&quot;&gt;1.0&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;/&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;c&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;log_mean_exp&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;That’s the entire numerical core of the method: draw \(N\) Gaussian samples around a
query point, evaluate a terminal cost, log-sum-exp them. Everything else — the Picard
loop, the gradient estimator, multi-GPU sharding via &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;GPUDistributor&lt;/code&gt; — is bookkeeping
around this one kernel. Defining a new problem means writing a Hamiltonian and a
terminal cost, e.g. the two-rocket pursuit-evasion game from
&lt;a href=&quot;https://github.com/robotsorcerer/levelsetpy/blob/main/monte_carlo/examples/ex_rockets_3d_comparison.py&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;&lt;code&gt;examples/ex_rockets_3d_comparison.py&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;k&quot;&gt;class&lt;/span&gt; &lt;span class=&quot;nc&quot;&gt;RocketsRelativeLSHamiltonian&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Hamiltonian&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;):&lt;/span&gt;
    &lt;span class=&quot;s&quot;&gt;&quot;&quot;&quot;H = -a·cos(x3)·p1 + (g-a-a·sin(x3))·p2 + u_bound²|p1·x1+p3| - u_bound²|p2·x1+p3|&quot;&quot;&quot;&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;__call__&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;t&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;x&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;p&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;):&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;p1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;p2&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;p3&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;p&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[...,&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;],&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;p&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[...,&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;],&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;p&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[...,&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;x1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;x3&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;x&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[...,&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;],&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;x&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[...,&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;smooth_abs&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;lambda&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;z&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;jnp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;sqrt&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;z&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;**&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;eps&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;**&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;a&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;jnp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;cos&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;x3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;p1&lt;/span&gt;
                &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;grav&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;a&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;a&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;jnp&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;sin&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;x3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;p2&lt;/span&gt;
                &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;u_eff&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;smooth_abs&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;p1&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;x1&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;p3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
                &lt;span class=&quot;o&quot;&gt;-&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;u_eff&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;smooth_abs&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;p2&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;x1&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;p3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;5-what-the-theory-actually-guarantees&quot;&gt;&lt;strong&gt;5. What the theory actually guarantees&lt;/strong&gt;&lt;/h3&gt;

&lt;p&gt;Two results anchor the paper, and it’s worth stating plainly what they do and don’t
claim.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Finite-sample concentration.&lt;/strong&gt; Fix a query state and a frozen coefficient \(c &amp;gt; 0\).
With \(N\) i.i.d. Gaussian draws, \(\hat v_{c,N}\) concentrates around the true frozen
expectation at a Hoeffding rate — \(O(N^{-1/2})\), the standard Monte Carlo law (full
proof in &lt;strong&gt;&lt;a href=&quot;#appendix-b-finite-sample-concentration-proof&quot;&gt;Appendix B&lt;/a&gt;&lt;/strong&gt;). The explicit
sample-size corollary gives&lt;/p&gt;

\[\begin{align}
N \;\ge\; \frac{(\beta-\alpha)^2}{2\alpha^2\,(1-e^{-c\varepsilon})^2}\,\log\!\Big(\frac{2}{\alpha}\Big),
\qquad \alpha = e^{-c\,g_{\max}},\;\; \beta = e^{-c\,g_{\min}}. \label{eq:sample-size}
\end{align}\]

&lt;p&gt;Here’s the honest part, which a NeurIPS reviewer caught and which is now stated
explicitly in the paper’s limitations: with \(c = 1/\delta\), this bound scales as
\(N \gtrsim \exp\!\big(2(g_{\max}-g_{\min})/\delta\big)\) — &lt;strong&gt;exponential&lt;/strong&gt; in
\(1/\delta\) — directly opposed to wanting small \(\delta\) for better viscosity
accuracy (\(O(\sqrt\delta)\), Crandall-Lions). Taken literally, for the rocket/Dubins
geometry the bound’s exponent is in the hundreds. What keeps the method usable in
practice: the bound is a worst-case Hoeffding estimate that ignores variance, and the
variance is empirically well-behaved — see §6. There is currently no known general class
of Hamiltonians for which the bound’s constants are controlled uniformly in \(\delta\);
that’s an open problem, not a solved one, and the paper says so.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Picard contraction.&lt;/strong&gt; Under Lipschitz/nondegeneracy conditions on the reconstructed
gradient (roughly: \(|Dv|\) bounded away from zero, the co-state direction not too
erratic), the frozen-coefficient update map \(\Lambda = \Phi \circ \Gamma\) is a
contraction, so Algorithm 1 converges linearly to a unique fixed point. That fixed point
is &lt;strong&gt;not&lt;/strong&gt;, in general, the true viscosity solution — the gap is the quasi-linearization
defect from §3, bounded separately by a Duhamel-norm argument. Two error sources, two
theorems, and the paper is careful not to conflate them.&lt;/p&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;6-does-it-actually-work-rockets-and-dubins-validated-properly&quot;&gt;&lt;strong&gt;6. Does it actually work? Rockets and Dubins, validated properly&lt;/strong&gt;&lt;/h3&gt;

&lt;p&gt;Numbers you can’t independently check are not numbers. Every claim in this section comes
from &lt;a href=&quot;https://github.com/robotsorcerer/levelsetpy/blob/main/monte_carlo/examples/stats_bonferroni_holm.py&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;&lt;code&gt;examples/stats_bonferroni_holm.py&lt;/code&gt;&lt;/a&gt; — 30 independent
Monte Carlo seeds per condition, Holm-Bonferroni-corrected significance tests, benchmarked
against &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;LevelSetPy&lt;/code&gt;’s grid-based finite-difference solver as ground truth.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Table 1.&lt;/strong&gt; Rockets and Dubins pursuit-evasion error metrics against the grid-based
reference, mean \(\pm\) one standard deviation over 30 independent Monte Carlo seeds.&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;System&lt;/th&gt;
      &lt;th&gt;\(\theta\) (rad)&lt;/th&gt;
      &lt;th&gt;\(L^\infty\)&lt;/th&gt;
      &lt;th&gt;\(L^2_{\text{rel}}\)&lt;/th&gt;
      &lt;th&gt;MC time (s)&lt;/th&gt;
      &lt;th&gt;Iters&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;Rockets&lt;/td&gt;
      &lt;td&gt;\(-\pi/2\)&lt;/td&gt;
      &lt;td&gt;\(0.855 \pm 0.047\)&lt;/td&gt;
      &lt;td&gt;\(0.098 \pm 0.002\)&lt;/td&gt;
      &lt;td&gt;\(13.5 \pm 0.1\)&lt;/td&gt;
      &lt;td&gt;12&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Rockets&lt;/td&gt;
      &lt;td&gt;\(0\)&lt;/td&gt;
      &lt;td&gt;\(1.034 \pm 0.067\)&lt;/td&gt;
      &lt;td&gt;\(0.101 \pm 0.001\)&lt;/td&gt;
      &lt;td&gt;\(13.5 \pm 0.1\)&lt;/td&gt;
      &lt;td&gt;12&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Rockets&lt;/td&gt;
      &lt;td&gt;\(\pi/2\)&lt;/td&gt;
      &lt;td&gt;\(0.895 \pm 0.109\)&lt;/td&gt;
      &lt;td&gt;\(0.090 \pm 0.002\)&lt;/td&gt;
      &lt;td&gt;\(13.6 \pm 0.1\)&lt;/td&gt;
      &lt;td&gt;12&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Dubins&lt;/td&gt;
      &lt;td&gt;\(-\pi/2\)&lt;/td&gt;
      &lt;td&gt;\(1.352 \pm 0.002\)&lt;/td&gt;
      &lt;td&gt;\(0.131 \pm 0.003\)&lt;/td&gt;
      &lt;td&gt;\(23.6 \pm 0.1\)&lt;/td&gt;
      &lt;td&gt;15&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Dubins&lt;/td&gt;
      &lt;td&gt;\(0\)&lt;/td&gt;
      &lt;td&gt;\(0.701 \pm 0.144\)&lt;/td&gt;
      &lt;td&gt;\(0.024 \pm 0.001\)&lt;/td&gt;
      &lt;td&gt;\(23.7 \pm 0.1\)&lt;/td&gt;
      &lt;td&gt;15&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Dubins&lt;/td&gt;
      &lt;td&gt;\(\pi/2\)&lt;/td&gt;
      &lt;td&gt;\(1.352 \pm 0.002\)&lt;/td&gt;
      &lt;td&gt;\(0.132 \pm 0.003\)&lt;/td&gt;
      &lt;td&gt;\(23.7 \pm 0.1\)&lt;/td&gt;
      &lt;td&gt;15&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;For every one of these six conditions, a one-sided one-sample \(t\)-test rejects
\(L^2_{\text{rel}} \ge \sqrt{\delta}\) (the Crandall-Lions viscosity bound) at
\(p_{\text{holm}} &amp;lt; 10^{-50}\) — the sampling-plus-iteration error is &lt;em&gt;statistically&lt;/em&gt;,
not just numerically, well inside the theoretical budget. A second, stricter test
tells a more honest story: a paired Wilcoxon signed-rank test of the 30-seed-averaged MC
field against the grid reference &lt;em&gt;also&lt;/em&gt; rejects equality everywhere
(\(p_{\text{holm}} &amp;lt; 10^{-8}\)) — averaging away sampling noise does &lt;strong&gt;not&lt;/strong&gt; make the two
fields agree, because a systematic quasi-linearization defect (§3, §5) remains. Both
things are true at once: the method is accurate, and it is not exact.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/hjgauss/rockets_3d_slices.jpg&quot; alt=&quot;Rockets BRT: grid solver vs Monte Carlo, with pointwise error&quot; style=&quot;max-width:100%;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Top: &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;LevelSetPy&lt;/code&gt;’s grid solution. Middle: HJ-Gauss (Monte Carlo). Bottom: pointwise
error, largest near the zero level-set boundary where \(|Dv^\delta|\) is largest — exactly
where the theory in §3 and §5 says the quasi-linearization defect should concentrate.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The apparent asymmetry between \(\theta = -90°\) and \(\theta = +90°\) above is physical,
not numerical: the relative-dynamics Hamiltonian carries a gravity term
\((g - a - a\sin\theta)\) in the vertical channel, giving effective drift \(g - 2a\) at
\(+90°\) but \(g\) at \(-90°\). A Holm-Bonferroni-corrected Mann-Whitney test confirms
this is statistically significant for Rockets (\(p_{\text{holm}} &amp;lt; 10^{-9}\)) but &lt;em&gt;not&lt;/em&gt;
for the gravity-free Dubins vehicle (\(p_{\text{holm}} = 0.53\)) — the statistics track
the physics, not an estimator artifact.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/hjgauss/dubins_3d_comparison.jpg&quot; alt=&quot;Dubins BRT: grid solver vs Monte Carlo&quot; style=&quot;max-width:100%;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/hjgauss/rockets_3d_brt.jpg&quot; alt=&quot;3D backward reachable tube isosurface, grid vs Monte Carlo&quot; style=&quot;max-width:100%;&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The full 3D backward reachable tube, extracted via marching cubes over a \(25^3\)
evaluation grid — a genuinely 3D certificate, not a stack of 2D slices.&lt;/em&gt;&lt;/p&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;7-going-high-dimensional-15-rockets-45-states&quot;&gt;&lt;strong&gt;7. Going high-dimensional: 15 rockets, 45 states&lt;/strong&gt;&lt;/h3&gt;

&lt;p&gt;The whole point of replacing a grid with samples is that it stops caring about
dimension. &lt;a href=&quot;https://github.com/robotsorcerer/levelsetpy/blob/main/monte_carlo/examples/ex_multiagent_scalability.py&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;&lt;code&gt;examples/ex_multiagent_scalability.py&lt;/code&gt;&lt;/a&gt; sets up a 15-rocket,
single-evader pursuit-evasion game — state \(x = (x_1,y_1,\theta_1,\ldots,x_{15},y_{15},\theta_{15}) \in \mathbb{R}^{45}\).
A \(101\)-point-per-axis grid at \(n=45\) would need \(101^{45} \approx 10^{90}\) cells;
the sampler needs \(O(N\cdot n) \approx 7.2\) MB per iteration, independent of the
exponent.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Table 2.&lt;/strong&gt; 15-rocket, 45-dimensional multi-agent scalability across three speed
regimes, mean \(\pm\) one standard deviation over 30 independent Monte Carlo seeds. No
grid-based reference exists at this dimension (see discussion below).&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;Case&lt;/th&gt;
      &lt;th&gt;\(a_{\text{evader}}\)&lt;/th&gt;
      &lt;th&gt;\(a_{\text{pursuers}}\)&lt;/th&gt;
      &lt;th&gt;Iters&lt;/th&gt;
      &lt;th&gt;Residual \(\varepsilon(k)\)&lt;/th&gt;
      &lt;th&gt;Wall-clock (s)&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;Evader faster&lt;/td&gt;
      &lt;td&gt;2.0&lt;/td&gt;
      &lt;td&gt;1.0&lt;/td&gt;
      &lt;td&gt;15&lt;/td&gt;
      &lt;td&gt;\(0.0002 \pm 0.0005\)&lt;/td&gt;
      &lt;td&gt;\(12.7 \pm 0.2\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Equal speed&lt;/td&gt;
      &lt;td&gt;1.0&lt;/td&gt;
      &lt;td&gt;1.0&lt;/td&gt;
      &lt;td&gt;15&lt;/td&gt;
      &lt;td&gt;\(0.0003 \pm 0.0005\)&lt;/td&gt;
      &lt;td&gt;\(12.6 \pm 0.1\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Pursuers faster&lt;/td&gt;
      &lt;td&gt;1.0&lt;/td&gt;
      &lt;td&gt;2.0&lt;/td&gt;
      &lt;td&gt;15&lt;/td&gt;
      &lt;td&gt;\(0.0006 \pm 0.0013\)&lt;/td&gt;
      &lt;td&gt;\(12.6 \pm 0.1\)&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;No grid-based reference exists at \(n=45\) to check accuracy against — the table
reports what &lt;em&gt;can&lt;/em&gt; be measured honestly (iteration stability, memory, wall-clock), and
that’s exactly how the paper frames it: this demonstrates &lt;em&gt;scalability&lt;/em&gt;, not certified
accuracy. One thing worth noting: the residual floor here (about \(0.0002\)-\(0.0006\))
sits roughly &lt;strong&gt;two orders of magnitude below&lt;/strong&gt; the 3D benchmarks’ floor
(\(0.02\)-\(0.06\)) — not because higher dimension is somehow easier, but because the
45D evaluation states are drawn from a domain large enough that almost none of them land
near the coefficient-turnover region where the quasi-linearization defect concentrates.
Pairwise Holm-Bonferroni-corrected tests find no significant difference in the residual
floor across the three speed regimes (\(p_{\text{holm}} \ge 0.93\) for all three pairs) —
consistent with the floor being set by \((N,\delta)\), not by the game’s own parameters.&lt;/p&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;8-the-stress-test-100000-starlings&quot;&gt;&lt;strong&gt;8. The stress test: 100,000 starlings&lt;/strong&gt;&lt;/h3&gt;

&lt;p&gt;The most demanding application in the paper isn’t high-dimensional in the usual sense —
it’s high-&lt;em&gt;cardinality&lt;/em&gt;. Starling murmurations are modeled as 4D aerial Dubins vehicles
under predator attack, and the safety question is: does the flock’s reachable-unsafe set
change topology as an attack unfolds? The murmuration is partitioned into flocks, each
resolved by its own value function; the population enters only through embarrassingly
parallel per-bird certification once that value function exists — so certifying
\(100{,}000\) birds costs the same &lt;em&gt;solve&lt;/em&gt; as certifying one, plus \(100{,}000\) batched
forward evaluations.&lt;/p&gt;

&lt;p&gt;The interesting part is what the &lt;em&gt;topology&lt;/em&gt; of the zero sublevel set tells you. Three
integers per time step — Euler characteristic \(\chi\), first Betti number \(\beta_1\),
connected-component count \(n_{\text{comp}}\) — recover the field-documented repertoire
of collective evasive behaviors as topological events: &lt;strong&gt;vacuole nucleation&lt;/strong&gt; (a drop in
\(\chi\) when a predator penetrates the flock), a defensive &lt;strong&gt;cordon&lt;/strong&gt; (an annular safe
set, \(\beta_1 = 1\), collapsing to \(\beta_1 = 0\) as the predator closes in), and
&lt;strong&gt;flock fragmentation&lt;/strong&gt; (a rising component count). It’s a genuinely different way to
report a safety certificate: not just &lt;em&gt;whether&lt;/em&gt; safety is being lost, but &lt;em&gt;how&lt;/em&gt;.&lt;/p&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;9-where-it-doesnt-work&quot;&gt;&lt;strong&gt;9. Where it doesn’t work&lt;/strong&gt;&lt;/h3&gt;

&lt;p&gt;The paper is candid about scope, and it’s worth repeating that candor here rather than
letting the headline numbers speak alone:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;Non-degeneracy is load-bearing.&lt;/strong&gt; The frozen coefficient \(c = 2H/(\delta|Dv|^2)\)
needs \(|Dv|\) bounded away from zero; regularizing the denominator where it degenerates
costs a quantified, uniform perturbation to the effective Hamiltonian — but it &lt;em&gt;is&lt;/em&gt; a
cost, paid near flat interiors and shocks.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;The sample bound is exponentially loose in \(1/\delta\)&lt;/strong&gt; (§5) — a real, currently
unresolved tension between wanting small \(\delta\) and wanting small \(N\).&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Gradient estimation noise&lt;/strong&gt; compounds in high-viscosity regimes or when samples
concentrate in narrow regions.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;No independent ground truth exists at \(n=45\).&lt;/strong&gt; The multi-agent result demonstrates
scalability; it cannot demonstrate certified accuracy, because nothing else can solve
that problem to compare against.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Everything above is CPU-only, single-core&lt;/strong&gt;, on purpose (to make the memory-frugality
claim honest), leaving 19 cores and an idle GPU on the table — multi-core and GPU
execution should help, but that scaling hasn’t been benchmarked yet.&lt;/li&gt;
&lt;/ul&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;appendix-a-the-cole-hopf-reduction-in-full&quot;&gt;Appendix A: The Cole-Hopf Reduction in Full&lt;/h3&gt;

&lt;p&gt;Section 2 asserted that \(\omega^\delta := \exp(-v^\delta/\delta)\) satisfies the plain
heat equation exactly when \(H = \tfrac{1}{2}|p|^2\). Here is the full computation, not
just the claim.&lt;/p&gt;

&lt;p&gt;Write \(c\) for a constant to be determined and \(\omega^\delta := \exp(-c\,v^\delta)\).
By the chain rule,&lt;/p&gt;

\[\begin{align}
\omega^\delta_t &amp;amp;= -c\,v^\delta_t\,\omega^\delta, \label{eq:a-omega-t}\\
D\omega^\delta &amp;amp;= -c\,Dv^\delta\,\omega^\delta, \label{eq:a-omega-grad}\\
\Delta\omega^\delta &amp;amp;= \operatorname{div}(D\omega^\delta)
= \operatorname{div}\!\big(-c\,Dv^\delta\,\omega^\delta\big)
= \big(c^2|Dv^\delta|^2 - c\,\Delta v^\delta\big)\,\omega^\delta. \label{eq:a-omega-lap}
\end{align}\]

&lt;p&gt;Substituting \eqref{eq:a-omega-t} and \eqref{eq:a-omega-lap} into
\(\omega^\delta_t - \tfrac{\delta}{2}\Delta\omega^\delta\):&lt;/p&gt;

\[\begin{align}
\omega^\delta_t - \frac{\delta}{2}\Delta\omega^\delta
&amp;amp;= -c\,v^\delta_t\,\omega^\delta - \frac{\delta}{2}\big(c^2|Dv^\delta|^2 - c\,\Delta v^\delta\big)\,\omega^\delta \nonumber\\
&amp;amp;= c\,\omega^\delta\left[-v^\delta_t - \frac{\delta}{2}\,c\,|Dv^\delta|^2 + \frac{\delta}{2}\Delta v^\delta\right]. \label{eq:a-substituted}
\end{align}\]

&lt;p&gt;Since \(c\,\omega^\delta \neq 0\), the bracketed term in \eqref{eq:a-substituted}
vanishing is equivalent to&lt;/p&gt;

\[\begin{align}
v^\delta_t = \frac{\delta}{2}\Delta v^\delta - \frac{\delta}{2}\,c\,|Dv^\delta|^2. \label{eq:a-rearranged}
\end{align}\]

&lt;p&gt;Now compare \eqref{eq:a-rearranged} against the viscous HJ equation
\eqref{eq:viscous-hj} itself, rearranged the same way:
\(v^\delta_t = \tfrac{\delta}{2}\Delta v^\delta - H(t,x,Dv^\delta)\). The two agree
term-for-term exactly when&lt;/p&gt;

\[\begin{align}
\frac{\delta}{2}\,c\,|Dv^\delta|^2 = H(t,x,Dv^\delta). \label{eq:a-matching}
\end{align}\]

&lt;p&gt;For a &lt;strong&gt;general&lt;/strong&gt; Hamiltonian, \eqref{eq:a-matching} forces \(c\) to depend on
\((t,x)\) through \(Dv^\delta\) — which is exactly the spatially-varying coefficient
\(c(t,x)\) of Eq. \eqref{eq:frozen-coeff} in §3, and the reason a general \(H\) needs the
Picard quasi-linearization rather than a single exact substitution: \(c\) can no longer
be pulled out as a constant, so the chain-rule terms \(c_t\), \(Dc\), \(\Delta c\) that
we implicitly assumed away above resurface as a residual (see the paper’s Lemma on the
exact residual of the generalized Cole-Hopf transform for the full accounting of that
residual).&lt;/p&gt;

&lt;p&gt;For the &lt;strong&gt;quadratic&lt;/strong&gt; case \(H = \tfrac{1}{2}|p|^2\), however, \eqref{eq:a-matching}
becomes \(\tfrac{\delta}{2}\,c\,|Dv^\delta|^2 = \tfrac{1}{2}|Dv^\delta|^2\), which solves
to&lt;/p&gt;

\[\begin{align}
c = \frac{1}{\delta}, \label{eq:a-c-solves}
\end{align}\]

&lt;p&gt;a genuine &lt;strong&gt;constant&lt;/strong&gt;, independent of \((t,x)\) — self-consistently justifying the
assumption that \(c\) could be pulled out of the derivatives in
\eqref{eq:a-omega-t}-\eqref{eq:a-omega-lap} in the first place. With
\(c = 1/\delta\), \(\omega^\delta = \exp(-v^\delta/\delta)\) satisfies the heat equation
\(\omega^\delta_t = \tfrac{\delta}{2}\Delta\omega^\delta\) &lt;strong&gt;exactly&lt;/strong&gt;, with initial data
\(\omega^\delta(0,x) = \exp(-v^\delta(0,x)/\delta) = \exp(-c\,g(x))\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;From heat equation to Gaussian expectation.&lt;/strong&gt; The equation
\(\omega_t = \tfrac{\delta}{2}\Delta\omega\) on \(\mathbb{R}^n \times (0,T]\) with
initial data \(\omega(0,y) = \exp(-c\,g(y))\) has the classical Green’s-function
(convolution) solution&lt;/p&gt;

\[\begin{align}
\omega(t,x) = \int_{\mathbb{R}^n} K_{\delta t}(x-y)\,\exp\big(-c\,g(y)\big)\,dy,
\qquad K_{\sigma^2}(z) := \frac{1}{(2\pi\sigma^2)^{n/2}}\exp\!\left(-\frac{|z|^2}{2\sigma^2}\right), \label{eq:a-greens}
\end{align}\]

&lt;p&gt;which is standard for \(u_t = D\Delta u\) with fundamental solution
\((4\pi Dt)^{-n/2}\exp(-|x|^2/(4Dt))\), here with \(D = \delta/2\) giving exactly
\(\sigma^2 = \delta t\) in \eqref{eq:a-greens}. The kernel
\(K_{\delta t}(x-y)\) in \eqref{eq:a-greens} is, by inspection, precisely the probability
density of \(Y \sim \mathcal{N}(x, \delta t\,I_n)\) evaluated at \(y\) — so the
convolution integral is &lt;em&gt;by definition&lt;/em&gt; the expectation&lt;/p&gt;

\[\begin{align}
\omega(t,x) = \mathbb{E}_{y \sim \mathcal{N}(x,\,\delta t I_n)}\Big[\exp\big(-c\,g(y)\big)\Big]. \label{eq:a-feynman-kac}
\end{align}\]

&lt;p&gt;Finally, undoing the substitution, \(v^\delta(t,x) = -\tfrac{1}{c}\log\omega(t,x)\)
together with \eqref{eq:a-feynman-kac} recovers Eq. \eqref{eq:value-expectation} of §2
exactly. \(\blacksquare\)&lt;/p&gt;

&lt;h3 id=&quot;appendix-b-finite-sample-concentration-proof&quot;&gt;Appendix B: Finite-Sample Concentration Proof&lt;/h3&gt;

&lt;p&gt;Section 5 stated a finite-sample concentration bound for the plug-in estimator without
proof. Here is the derivation, fixing a query state and a frozen coefficient \(c &amp;gt; 0\)
throughout (so \(c\) is treated as a known constant, not re-derived from data).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Setup.&lt;/strong&gt; Let \(\zeta \sim \mathcal{N}(x, \delta t\,I_n)\) and assume the terminal cost
is bounded: \(g_{\min} \le g(\zeta) \le g_{\max}\) almost surely. Define
\(Z := \exp(-c\,g(\zeta))\), \(\mu := \mathbb{E}[Z]\), and the true frozen value
\(v_c := -\tfrac{1}{c}\log\mu\). Since \(c&amp;gt;0\) and \(\exp(-c\cdot)\) is decreasing,
boundedness of \(g\) gives boundedness of \(Z\):&lt;/p&gt;

\[\begin{align}
\alpha \le Z \le \beta \quad \text{a.s.}, \qquad \alpha := e^{-c\,g_{\max}},\;\; \beta := e^{-c\,g_{\min}}. \label{eq:b-bounds}
\end{align}\]

&lt;p&gt;Given i.i.d. draws \(\zeta_1,\ldots,\zeta_N\), let \(Z_i := \exp(-c\,g(\zeta_i))\),
\(\bar Z_N := \tfrac{1}{N}\sum_i Z_i\), and the plug-in estimator
\(\hat v_{c,N} := -\tfrac{1}{c}\log\bar Z_N\). We want to bound
\(\mathbb{P}(|\hat v_{c,N} - v_c| \ge \varepsilon)\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Step 1: Hoeffding on the raw average.&lt;/strong&gt; Since \(Z_1,\ldots,Z_N\) are i.i.d. and bounded
in \([\alpha,\beta]\) by \eqref{eq:b-bounds}, Hoeffding’s inequality gives, for any \(s&amp;gt;0\),&lt;/p&gt;

\[\begin{align}
\mathbb{P}\big(\bar Z_N - \mu \ge s\big) \le \exp\!\left(-\frac{2Ns^2}{(\beta-\alpha)^2}\right),
\qquad
\mathbb{P}\big(\bar Z_N - \mu \le -s\big) \le \exp\!\left(-\frac{2Ns^2}{(\beta-\alpha)^2}\right). \label{eq:b-hoeffding}
\end{align}\]

&lt;p&gt;&lt;strong&gt;Step 2: translate the log-deviation event.&lt;/strong&gt; By definition,
\(\hat v_{c,N} - v_c = -\tfrac{1}{c}\log(\bar Z_N/\mu)\), so&lt;/p&gt;

\[\begin{align}
|\hat v_{c,N} - v_c| \ge \varepsilon
\iff \bar Z_N \ge \mu\, e^{c\varepsilon} \;\; \text{or} \;\; \bar Z_N \le \mu\, e^{-c\varepsilon}
\iff \bar Z_N - \mu \ge \mu(e^{c\varepsilon}-1) \;\; \text{or} \;\; \bar Z_N - \mu \le -\mu(1-e^{-c\varepsilon}). \label{eq:b-event}
\end{align}\]

&lt;p&gt;&lt;strong&gt;Step 3: combine via a union bound.&lt;/strong&gt; For every \(\varepsilon \ge 0\),
\(e^{c\varepsilon}-1 \ge 1-e^{-c\varepsilon}\) (both sides vanish at \(\varepsilon=0\)
and the derivative of their difference, \(c(e^{c\varepsilon}+e^{-c\varepsilon}-2)\cdot\tfrac12\)-type
terms, is nonnegative for \(\varepsilon \ge 0\)) — so the first event in
\eqref{eq:b-event} has an equal or smaller probability, under \eqref{eq:b-hoeffding},
than if it used the smaller threshold \(\mu(1-e^{-c\varepsilon})\) as well. Applying
\eqref{eq:b-hoeffding} to both events in \eqref{eq:b-event} with
\(s = \mu(1-e^{-c\varepsilon})\) and summing (union bound):&lt;/p&gt;

\[\begin{align}
\mathbb{P}\big(|\hat v_{c,N} - v_c| \ge \varepsilon\big)
\;\le\; 2\exp\!\left(-\frac{2N\mu^2\big(1-e^{-c\varepsilon}\big)^2}{(\beta-\alpha)^2}\right). \label{eq:b-final}
\end{align}\]

&lt;p&gt;This is exactly the concentration bound quoted in §5. \(\blacksquare\)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Corollary (explicit sample size).&lt;/strong&gt; Since \(\mu \ge \alpha\) pointwise, replacing
\(\mu\) by its lower bound \(\alpha\) in \eqref{eq:b-final} only weakens it, so requiring
the (now \(\mu\)-free) right-hand side to be at most \(\alpha\) and solving for \(N\)
recovers the explicit sample-size guarantee, Eq. \eqref{eq:sample-size}, guaranteeing
\(\mathbb{P}(|\hat v_{c,N}-v_c|\ge\varepsilon) \le \alpha\).&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;further-reading&quot;&gt;Further reading&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;Paper:&lt;/strong&gt; &lt;a href=&quot;/downloads/Papers/hjgauss.pdf&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;Sampling-based HJ Reachability — 28-page compilation (PDF)&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Animations:&lt;/strong&gt; &lt;a href=&quot;/downloads/Papers/assets/anim/roysoc_rockets.gif&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;Rockets&lt;/a&gt; · &lt;a href=&quot;/downloads/Papers/assets/anim/roysoc_dubins.gif&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;Dubins vehicles&lt;/a&gt; · &lt;a href=&quot;/downloads/Papers/assets/anim/roysoc_double_integrator.gif&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;Double integrator&lt;/a&gt; · &lt;a href=&quot;/downloads/Papers/assets/anim/roysoc_murmuration.gif&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;Starling model&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Code:&lt;/strong&gt; &lt;a href=&quot;https://github.com/robotsorcerer/levelsetpy/tree/main/levelsetpy/monte_carlo&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;github.com/robotsorcerer/levelsetpy/tree/main/levelsetpy/monte_carlo&lt;/a&gt; —
see &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;src/hj_sampler.py&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;src/heat_solver.py&lt;/code&gt; for the solver, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;examples/&lt;/code&gt; for every
benchmark in this post, and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;examples/stats_bonferroni_holm.py&lt;/code&gt; for the statistical
validation sweep.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Grid-based reference:&lt;/strong&gt; &lt;a href=&quot;https://github.com/robotsorcerer/levelsetpy&quot; target=&quot;_blank&quot; rel=&quot;noopener noreferrer&quot;&gt;LevelSetPy&lt;/a&gt;,
the finite-difference solver used as ground truth throughout.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class=&quot;footnotes&quot; role=&quot;doc-endnotes&quot;&gt;
  &lt;ol&gt;
    &lt;li id=&quot;fn:reliability&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Reliability is taken to mean guaranteed consistency in system performance over time. &lt;a href=&quot;#fnref:reliability&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:verification&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;That which entails generating evidence that a system, or any of its components, satisfy all specified requirements and functional and allocated baselines is termed verification (DAU). &lt;a href=&quot;#fnref:verification&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
    &lt;li id=&quot;fn:validation&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Validation entails providing evidence that system capabilities comply with an end-user’s performance requirements and satisfy its intended operational environment’s specifications (DAU). &lt;a href=&quot;#fnref:validation&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
  &lt;/ol&gt;
&lt;/div&gt;
</description>
        <pubDate>Sun, 26 Jul 2026 09:00:00 +0000</pubDate>
        <link>http://scriptedonachip.com/hjgauss</link>
        <guid isPermaLink="true">http://scriptedonachip.com/hjgauss</guid>
        
        
        <category>reachability</category>
        
        <category>control</category>
        
        <category>robotics</category>
        
        <category>optimal-control</category>
        
        <category>hamilton-jacobi</category>
        
      </item>
    
      <item>
        <title>Kolmogorov Regression for Robust Diffusion Policies</title>
        <description>&lt;h1 id=&quot;mr-kolmogorov-reinforces-diffusion&quot;&gt;Mr. Kolmogorov Reinforces Diffusion&lt;/h1&gt;

&lt;p&gt;This exposition develops the backward Kolmogorov equation (BKE) for diffusion processes
on an infinite-dimensional Hilbert space \(\mathcal{H}\), with particular attention to
the Ornstein-Uhlenbeck (OU) process that underlies modern diffusion generative models.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;This note is a theoretical compendium to the results presented in the paper: &lt;a href=&quot;/downloads/Papers/robodiff.pdf&quot;&gt;&lt;strong&gt;Kolmogorov Reggression for Robust Diffusion Policies&lt;/strong&gt;&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The exposition is built from first principles: the spectral structure of the covariance
operator (Mercer) determines the trace-class condition, which characterises the
Cameron-Martin space, which in turn fixes the class of admissible Wiener processes —
and from there the generator and BKE follow directly.&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;1-mercers-theorem-for-the-covariance-operator-mathcalc_mu&quot;&gt;1. Mercer’s Theorem for the Covariance Operator \(\mathcal{C}_\mu\)&lt;/h2&gt;

&lt;p&gt;&lt;em&gt;Sources: Riesz–Sz.-Nagy, Functional Analysis (Frederick Ungar, 1955), §§97–98, pp. 242–246.&lt;/em&gt;&lt;/p&gt;
&lt;h3 id=&quot;setup&quot;&gt;Setup&lt;/h3&gt;

&lt;p&gt;Let \((\mathcal{X}, d)\) be a compact metric space and \(\mu\) a Borel probability measure
on \(\mathcal{H} = L^2(\mathcal{X}, \mu)\) with \(\int_\mathcal{H} \|x\|^2_\mathcal{H}\,\mu(dx) &amp;lt; \infty\).
Define the &lt;strong&gt;covariance operator&lt;/strong&gt; \(\mathcal{C}_\mu : \mathcal{H} \to \mathcal{H}\) by&lt;/p&gt;

\[\langle \mathcal{C}_\mu u,\, v \rangle_\mathcal{H}
= \int_\mathcal{H} \langle x, u \rangle_\mathcal{H}\,\langle x, v \rangle_\mathcal{H}\;\mu(dx),
\qquad u,v \in \mathcal{H},\]

&lt;p&gt;with associated &lt;strong&gt;covariance kernel&lt;/strong&gt; \(k : \mathcal{X} \times \mathcal{X} \to \mathbb{R}\),&lt;/p&gt;

\[k(x,y) = \int_\mathcal{H} \langle z, x \rangle_\mathcal{H}\,\langle z, y \rangle_\mathcal{H}\;\mu(dz),\]

&lt;p&gt;so that \((\mathcal{C}_\mu f)(x) = \int_\mathcal{X} k(x,y)\,f(y)\,\mu(dy)\).&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;&lt;strong&gt;Theorem (Mercer, 1909; Riesz–Sz.-Nagy §98).&lt;/strong&gt; &lt;em&gt;Suppose \(k\) is continuous and
symmetric, and \(\mathcal{C}_\mu\) is &lt;strong&gt;positive&lt;/strong&gt;, i.e.&lt;/em&gt;
\(\langle \mathcal{C}_\mu f, f \rangle_\mathcal{H} \ge 0\) &lt;em&gt;for all \(f \in \mathcal{H}\).
Then:&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;(i) Spectral decomposition.&lt;/strong&gt; There exists a countable orthonormal basis
\(\{e_k\}_{k=1}^\infty\) of \(\mathcal{H}\) and strictly positive eigenvalues
\(\lambda_1 \ge \lambda_2 \ge \cdots &amp;gt; 0\), \(\lambda_k \to 0\), such that&lt;/p&gt;

\[\mathcal{C}_\mu e_k = \lambda_k\, e_k\]

&lt;p&gt;and \(\mathcal{C}_\mu = \sum_{k=1}^\infty \lambda_k\,\langle\cdot, e_k\rangle_\mathcal{H}\, e_k\)
in the strong operator topology.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;(ii) Uniform kernel expansion.&lt;/strong&gt; The series&lt;/p&gt;

\[k(x, y) = \sum_{k=1}^\infty \lambda_k\, e_k(x)\, e_k(y)\]

&lt;p&gt;converges &lt;strong&gt;absolutely and uniformly&lt;/strong&gt; on \(\mathcal{X} \times \mathcal{X}\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;(iii) Diagonal and trace.&lt;/strong&gt; On the diagonal, uniformly in \(x\):
\(k(x,x) = \sum_{k=1}^\infty \lambda_k\,|e_k(x)|^2.\)
Integrating over \(\mathcal{X}\) — exchange of sum and integral justified by Dini’s
theorem, since the partial sums of positive continuous functions converge uniformly to
the continuous function \(k(x,x)\) — yields the &lt;strong&gt;trace identity&lt;/strong&gt;:&lt;/p&gt;

\[\operatorname{Tr}[\mathcal{C}_\mu]
= \int_\mathcal{X} k(x,x)\,\mu(dx)
= \sum_{k=1}^\infty \lambda_k &amp;lt; \infty,\]

&lt;p&gt;so positivity and continuity of \(k\) force \(\mathcal{C}_\mu\) to be &lt;strong&gt;trace-class&lt;/strong&gt;.&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;&lt;strong&gt;Remark (Hilbert–Schmidt vs. Mercer).&lt;/strong&gt; The Hilbert–Schmidt theorem (§97) gives the
\(L^2\)-mean-convergent expansion \(k = \sum_k \lambda_k e_k \otimes e_k\) for any
square-summable symmetric kernel without positivity.  Mercer upgrades this to uniform
convergence: positivity forces diagonal remainders \(k(x,x) - \sum_{i=1}^n \lambda_i|e_i(x)|^2 \ge 0\)
to be monotone decreasing; Dini’s theorem then promotes pointwise monotone convergence
of positive continuous functions to uniform convergence; Cauchy’s inequality propagates
this from the diagonal to the full kernel.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Corollary (best \(N\)-term approximation).&lt;/strong&gt; The partial sum
\(k_N = \sum_{i=1}^N \lambda_i e_i \otimes e_i\) uniquely minimises \(\|k - k_N\|_{L^2}\)
with residual \(\|k - k_N\|^2_{L^2} = \sum_{i&amp;gt;N}\lambda_i^2\), underpinning the
effective rank \(r_\mathrm{eff}(\mathcal{C}_\mu) = (\sum_k\lambda_k)^2/(\sum_k\lambda_k^2)\).&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;2-trace-class-operators&quot;&gt;2. Trace-Class Operators&lt;/h2&gt;

&lt;p&gt;The Mercer trace identity \(\sum_k \lambda_k &amp;lt; \infty\) is a spectral fact.  This section
gives it an intrinsic, basis-free meaning and connects it to Gaussian measure theory.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Basis-free definition.&lt;/strong&gt; For a non-negative symmetric operator
\(C : \mathcal{H} \to \mathcal{H}\), define&lt;/p&gt;

\[\operatorname{Tr}[C] := \sum_{k=1}^\infty \langle C e_k,\, e_k \rangle_{\mathcal{H}}\]

&lt;p&gt;for any complete orthonormal basis \(\{e_k\}\).  By Lidskii’s theorem, this sum is
independent of the choice of basis, so \(\operatorname{Tr}[C]\) is intrinsic.  We say
\(C\) is &lt;strong&gt;trace-class&lt;/strong&gt; if \(\operatorname{Tr}[C] &amp;lt; \infty\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Measure-theoretic characterization.&lt;/strong&gt; Let \(\mu = \mathcal{N}(0, C)\) be the Gaussian
measure on \(\mathcal{H}\) with covariance \(C\).  By definition of \(C\) as the covariance
operator of \(\mu\),&lt;/p&gt;

\[\langle C e_k,\, e_k \rangle_{\mathcal{H}} = \int_{\mathcal{H}} \langle x,\, e_k \rangle^2 \,\mu(dx).\]

&lt;p&gt;Summing over the basis and applying Parseval’s identity
\(\|x\|_\mathcal{H}^2 = \sum_k \langle x, e_k\rangle^2\) together with Tonelli’s theorem
(non-negative terms justify interchange of sum and integral):&lt;/p&gt;

\[\operatorname{Tr}[C]
= \sum_{k=1}^\infty \int_{\mathcal{H}} \langle x,\, e_k \rangle^2 \,\mu(dx)
= \int_{\mathcal{H}} \|x\|_{\mathcal{H}}^2 \,\mu(dx).\]

&lt;p&gt;Hence \(\operatorname{Tr}[C] &amp;lt; \infty\) is &lt;em&gt;equivalent&lt;/em&gt; to \(\mu\) having finite second
moment in \(\mathcal{H}\) — i.e., typical samples have finite \(\mathcal{H}\)-norm.
Choosing \(\{e_k\}\) to be the Mercer eigenbasis gives the full three-way equivalence:&lt;/p&gt;

\[\operatorname{Tr}[C]
\;=\; \int_{\mathcal{H}} \|x\|_{\mathcal{H}}^2\,\mu(dx)
\;=\; \sum_{k=1}^\infty \langle C e_k,\, e_k\rangle_{\mathcal{H}}
\;\triangleq\; \sum_{k=1}^\infty \lambda_k \;&amp;lt;\; \infty.\]

&lt;p&gt;The measure-theoretic form is the most fundamental; the basis-free operator form is
what appears in the BKE diffusion term; the eigenvalue sum is the most computationally
explicit.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Derived spectral identities.&lt;/strong&gt; The full hierarchy of norms and operators derived from
the Mercer eigenpairs \(\{(\lambda_k, e_k)\}\) is:&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;Quantity&lt;/th&gt;
      &lt;th&gt;Spectral form&lt;/th&gt;
      &lt;th&gt;Meaning&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;Action on \(x \in \mathcal{H}\)&lt;/td&gt;
      &lt;td&gt;\(\mathcal{C}_\mu x = \sum_k \lambda_k \langle x, e_k\rangle_\mathcal{H} e_k\)&lt;/td&gt;
      &lt;td&gt;decompose, scale, recompose&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Operator norm&lt;/td&gt;
      &lt;td&gt;\(|\mathcal{C}_\mu| = \lambda_1\)&lt;/td&gt;
      &lt;td&gt;largest stretch on unit ball in \(\mathcal{H}\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Hilbert–Schmidt norm&lt;/td&gt;
      &lt;td&gt;\(|\mathcal{C}_\mu|_{\mathrm{HS}}^2 = \sum_k \lambda_k^2\)&lt;/td&gt;
      &lt;td&gt;stronger than trace-class&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Trace (trace-class condition)&lt;/td&gt;
      &lt;td&gt;\(\operatorname{Tr}(\mathcal{C}_\mu) = \sum_k \lambda_k &amp;lt; \infty\)&lt;/td&gt;
      &lt;td&gt;finite expected energy&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;&lt;strong&gt;Square root&lt;/strong&gt;&lt;/td&gt;
      &lt;td&gt;\(\mathcal{C}_\mu^{1/2} e_k = \sqrt{\lambda_k}\, e_k\)&lt;/td&gt;
      &lt;td&gt;positive functional calculus&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Cameron-Martin norm&lt;/td&gt;
      &lt;td&gt;\(|h|^2_{\mathcal{H}_C} = \sum_k \hat{h}_k^2 / \lambda_k\)&lt;/td&gt;
      &lt;td&gt;inverse-square-root weighting&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;&lt;strong&gt;Note on the operator norm.&lt;/strong&gt; Unlike the Euclidean operator norm (largest singular value
of a matrix on \(\mathbb{R}^n\)), \(\|\mathcal{C}_\mu\| = \lambda_1\) is the induced norm
on \(\mathcal{H}\):&lt;/p&gt;

\[\|\mathcal{C}_\mu\| := \sup_{\|x\|_\mathcal{H} = 1} \|\mathcal{C}_\mu x\|_\mathcal{H}.\]

&lt;p&gt;For a self-adjoint positive operator the supremum is attained at \(e_1\) and equals
\(\lambda_1\).  The analogy with finite dimensions holds: for a symmetric positive
matrix \(A \in \mathbb{R}^{n\times n}\), the induced 2-norm is also the largest
eigenvalue.  The key difference is that in \(\mathcal{H}\) the “unit ball” is
infinite-dimensional.&lt;/p&gt;

&lt;p&gt;The norms form a strict hierarchy: \(\operatorname{Tr}(\mathcal{C}_\mu) &amp;lt; \infty\)
(trace-class) \(\Rightarrow\) \(\|\mathcal{C}_\mu\|_{\mathrm{HS}} &amp;lt; \infty\)
(Hilbert–Schmidt) \(\Rightarrow\) \(\|\mathcal{C}_\mu\| &amp;lt; \infty\) (bounded).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Where does \(\sqrt{\lambda_k}\) come from?&lt;/strong&gt;
The square root \(\mathcal{C}_\mu^{1/2}\) is the &lt;em&gt;unique positive operator&lt;/em&gt; satisfying
\((\mathcal{C}_\mu^{1/2})^2 = \mathcal{C}_\mu\).  On eigenvectors the spectral mapping
theorem forces&lt;/p&gt;

\[\mathcal{C}_\mu^{1/2} e_k = \sqrt{\lambda_k}\, e_k.\]

&lt;p&gt;This \(\sqrt{\lambda_k}\) is exactly the per-mode excitation rate of the
\(\mathcal{C}\)-Wiener process.  To see why, suppose
\(W_t^C = \sum_k c_k \beta_k(t) e_k\) for independent standard Brownian motions
\(\{\beta_k\}\) and coefficients \(c_k &amp;gt; 0\) to be determined.  The Gaussian increment
condition requires, for every \(h, g \in \mathcal{H}\),&lt;/p&gt;

\[\mathbb{E}\!\left[\langle W_t^C, h\rangle\langle W_t^C, g\rangle\right]
= t\,\langle \mathcal{C}_\mu h,\, g\rangle_{\mathcal{H}}.\]

&lt;p&gt;Taking \(h = g = e_j\) and using independence of the \(\beta_k\):&lt;/p&gt;

\[\mathbb{E}\!\left[\langle W_t^C, e_j\rangle^2\right]
= c_j^2\, t \stackrel{!}{=} t\,\langle \mathcal{C}_\mu e_j, e_j\rangle = t\lambda_j,\]

&lt;p&gt;so \(c_j = \sqrt{\lambda_j}\).  The series \(W_t^C = \sum_k \sqrt{\lambda_k}\,\beta_k(t)\,e_k\)
then converges in \(L^2(\Omega;\mathcal{H})\) if and only if&lt;/p&gt;

\[\sum_{k=1}^\infty \mathbb{E}\!\left[\lambda_k \beta_k(t)^2\right]
= t\sum_{k=1}^\infty \lambda_k = t\operatorname{Tr}(\mathcal{C}_\mu) &amp;lt; \infty,\]

&lt;p&gt;which is precisely the trace-class condition established above.  This closes the
logical chain: Mercer \(\to\) trace-class \(\to\) \(\sqrt{\lambda_k}\) coefficient \(\to\)
\(\mathcal{H}\)-valued Wiener process.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;&lt;strong&gt;Remark (Spectral representation).&lt;/strong&gt; The trace-class condition is necessary for the
series \(W_t^{\mathcal{C}} = \sum_{k=1}^\infty \sqrt{\lambda_k}\,\omega_k(t)\,e_k\)
to converge in \(L^2(\Omega;\mathcal{H})\), where \(\{\omega_k\}\) are independent
standard Brownian motions.  Each mode \(e_k\) is excited independently at rate
\(\sqrt{\lambda_k}\); the operator \(\mathcal{C}\) encodes the structure of covariance
noise across directions in \(\mathcal{H}\).&lt;/p&gt;
&lt;/blockquote&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;3-the-cameron-martin-space&quot;&gt;3. The Cameron-Martin Space&lt;/h2&gt;

&lt;p&gt;The Mercer eigenbasis \(\{(\lambda_k, e_k)\}\) of \(\mathcal{C}_\mu\) determines not just
the noise operator, but the geometry of directions along which the Gaussian measure
\(\mu = \mathcal{N}(0, C)\) can be shifted while remaining equivalent (absolutely
continuous).  These directions form the &lt;strong&gt;Cameron-Martin space&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Definition.&lt;/strong&gt; The Cameron-Martin space (equivalently, the reproducing kernel Hilbert
space, RKHS) of \(\mu\) is&lt;/p&gt;

\[\mathcal{H}_C := \mathrm{Range}\!\left(\mathcal{C}_\mu^{1/2}\right)
= \left\{h \in \mathcal{H} : \|h\|^2_{\mathcal{H}_C} &amp;lt; \infty \right\},\]

&lt;p&gt;equipped with the inner product&lt;/p&gt;

\[\langle h_1, h_2 \rangle_{\mathcal{H}_C}
:= \langle \mathcal{C}_\mu^{-1/2} h_1,\, \mathcal{C}_\mu^{-1/2} h_2 \rangle_{\mathcal{H}}.\]

&lt;p&gt;&lt;strong&gt;Spectral characterization.&lt;/strong&gt; In the Mercer eigenbasis, writing
\(\hat{h}_k := \langle h, e_k \rangle_\mathcal{H}\):&lt;/p&gt;

\[h \in \mathcal{H}_C
\iff \sum_{k=1}^\infty \frac{\hat{h}_k^2}{\lambda_k} &amp;lt; \infty,
\qquad \|h\|^2_{\mathcal{H}_C} = \sum_{k=1}^\infty \frac{\hat{h}_k^2}{\lambda_k}.\]

&lt;p&gt;Since \(\lambda_k \to 0\), the condition \(\sum_k \hat{h}_k^2/\lambda_k &amp;lt; \infty\) is
strictly stronger than \(h \in \mathcal{H}\) (which only requires \(\sum_k \hat{h}_k^2 &amp;lt; \infty\)),
so \(\mathcal{H}_C \subsetneq \mathcal{H}\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;RKHS interpretation.&lt;/strong&gt; The Mercer expansion \(k(x,y) = \sum_k \lambda_k e_k(x)e_k(y)\)
is precisely the reproducing kernel of \(\mathcal{H}_C\): for every \(x \in \mathcal{X}\),
\(k(\cdot, x) \in \mathcal{H}_C\) and \(\langle f, k(\cdot,x)\rangle_{\mathcal{H}_C} = f(x)\)
for all \(f \in \mathcal{H}_C\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Hilbert-Schmidt embedding.&lt;/strong&gt; The canonical inclusion
\(\iota : \mathcal{H}_C \hookrightarrow \mathcal{H}\) is Hilbert-Schmidt:&lt;/p&gt;

\[\|\iota\|^2_{\mathrm{HS}}
= \sum_{k=1}^\infty \|\iota e_k\|^2_{\mathcal{H}}
= \sum_{k=1}^\infty \lambda_k
= \operatorname{Tr}[\mathcal{C}_\mu] &amp;lt; \infty.\]

&lt;p&gt;The trace-class condition on \(\mathcal{C}_\mu\) is therefore &lt;em&gt;equivalent&lt;/em&gt; to the
embedding \(\mathcal{H}_C \hookrightarrow \mathcal{H}\) being Hilbert-Schmidt.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Cameron-Martin theorem.&lt;/strong&gt; For \(h \in \mathcal{H}\), the shifted measure
\(\mu_h(\cdot) := \mu(\cdot - h)\) satisfies:&lt;/p&gt;

\[\mu_h \ll \mu \iff h \in \mathcal{H}_C,\]

&lt;p&gt;with Radon-Nikodym derivative&lt;/p&gt;

\[\frac{d\mu_h}{d\mu}(x)
= \exp\!\left(\langle \mathcal{C}_\mu^{-1/2}h,\, \mathcal{C}_\mu^{-1/2}x\rangle_{\mathcal{H}}
- \tfrac{1}{2}\|h\|^2_{\mathcal{H}_C}\right).\]

&lt;p&gt;For \(h \notin \mathcal{H}_C\), the measures \(\mu_h\) and \(\mu\) are mutually singular.
The Cameron-Martin space is therefore the &lt;em&gt;exact&lt;/em&gt; set of admissible perturbations of
\(\mu\) — larger shifts destroy absolute continuity entirely.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Connection to the Cameron-Martin loss.&lt;/strong&gt; The operator \(\mathcal{C}_\mu^{-1/2}\)
appearing in the Radon-Nikodym derivative is the isometric embedding
\(\mathcal{H}_C \to \mathcal{H}\).  The denoising loss&lt;/p&gt;

\[\mathcal{L}_{\mathrm{CM}}(\theta)
= \mathbb{E}\!\left[\left\|C^{-1/2}\bigl(\eta_\theta(X_s, s) - \eta\bigr)\right\|_{\mathcal{H}}^2\right]\]

&lt;p&gt;measures the squared \(\mathcal{H}_C\)-norm of the prediction error: errors in
high-energy directions (large \(\lambda_k\)) are down-weighted by \(1/\lambda_k\),
achieving dimension-independent convergence rates (§10).&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;4-the-c-wiener-process&quot;&gt;4. The \(C\)-Wiener Process&lt;/h2&gt;

&lt;p&gt;With the trace-class condition established (§2) and the Cameron-Martin space in hand
(§3), we can now define the driving noise precisely.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Definition.&lt;/strong&gt; A &lt;strong&gt;\(C\)-Wiener process&lt;/strong&gt; is an \(\mathcal{H}\)-valued stochastic process
\(\{W_t^C\}_{t \ge 0}\) satisfying:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;\(W_0^C = 0\) almost surely.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Independent increments:&lt;/strong&gt; \(W_t^C - W_s^C \perp \mathcal{F}_s\) for all \(0 \le s \le t\).&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Gaussian increments:&lt;/strong&gt; for every \(h, g \in \mathcal{H}\),&lt;/li&gt;
&lt;/ol&gt;

\[\mathbb{E}\!\left[\langle W_t^C - W_s^C,\, h \rangle\,
\langle W_t^C - W_s^C,\, g \rangle\right]
= (t - s)\,\langle C h,\, g \rangle_{\mathcal{H}}.\]

&lt;p&gt;&lt;strong&gt;Spectral representation.&lt;/strong&gt; In the Mercer eigenbasis \(\{e_k\}\) of \(C\),&lt;/p&gt;

\[W_t^C = \sum_{k=1}^\infty \sqrt{\lambda_k}\, \beta_k(t)\, e_k,\]

&lt;p&gt;where \(\{\beta_k\}\) are independent standard Brownian motions.  The partial sums are a
Cauchy sequence in \(L^2(\Omega; \mathcal{H})\) if and only if
\(\sum_k \mathbb{E}[\lambda_k \beta_k(t)^2] = t\sum_k \lambda_k = t\operatorname{Tr}[C] &amp;lt; \infty\)
— exactly the trace-class condition of §2.  Each mode \(e_k\) is excited at rate
\(\sqrt{\lambda_k}\); the operator \(C\) encodes the covariance structure of noise across
directions of \(\mathcal{H}\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why trace-class is necessary.&lt;/strong&gt; If \(\operatorname{Tr}[C] = \infty\), the partial sums
diverge in \(\mathcal{H}\) and \(W_t^C\) does not take values in \(\mathcal{H}\) almost
surely.  By the Cameron-Martin theorem, paths would instead live in a larger
distribution space, and the Itô stochastic integral over \(\mathcal{H}\) would not be
well-defined.  The trace-class assumption is the minimal condition that keeps paths
\(\mathcal{H}\)-valued.&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;5-the-setup&quot;&gt;5. The Setup&lt;/h2&gt;

&lt;p&gt;The infinite-dimensional &lt;strong&gt;Ornstein-Uhlenbeck process&lt;/strong&gt; on \(\mathcal{H}\) is defined by
the Itô SDE&lt;/p&gt;

\[dX_t = -\tfrac{1}{2}X_t \, dt + dW_t^C, \qquad X_0 = x_0 \in \mathcal{H}.\]

&lt;p&gt;The drift \(-\tfrac{1}{2}X_t\) provides linear mean-reversion toward zero; the noise
\(dW_t^C\) injects energy in every direction \(e_k\) at rate \(\sqrt{\lambda_k}\).&lt;/p&gt;

&lt;p&gt;Given a measurable terminal functional \(f : \mathcal{H} \to \mathbb{R}\), define the
&lt;strong&gt;value function&lt;/strong&gt;&lt;/p&gt;

\[u(x, s) := \mathbb{E}[f(X_t) \mid X_s = x], \qquad s \le t.\]

&lt;p&gt;This is the conditional expectation of \(f\) evaluated at the terminal time \(t\), given
that the process occupies \(x\) at the earlier time \(s\).  The backward Kolmogorov equation
is the PDE that \(u\) satisfies as \(s\) varies.&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;6-the-infinitesimal-generator&quot;&gt;6. The Infinitesimal Generator&lt;/h2&gt;

&lt;p&gt;For a general \(\mathcal{H}\)-valued Itô process&lt;/p&gt;

\[dX_t = b(X_t)\,dt + dW_t^C,\]

&lt;p&gt;the &lt;strong&gt;infinitesimal generator&lt;/strong&gt; \(\mathcal{L}\) encodes the instantaneous rate of change
of expectations.  Acting on a smooth (Fréchet-differentiable) test function
\(\varphi : \mathcal{H} \to \mathbb{R}\), it reads&lt;/p&gt;

\[\mathcal{L}\varphi(x)
= \underbrace{\langle b(x),\, \nabla_x \varphi(x) \rangle_{\mathcal{H}}}_{\text{drift part}}
+ \underbrace{\tfrac{1}{2}\operatorname{Tr}\!\left[C \cdot D^2\varphi(x)\right]}_{\text{diffusion part}}.\]

&lt;p&gt;Here \(\nabla_x \varphi(x) \in \mathcal{H}\) is the Fréchet gradient and
\(D^2\varphi(x) : \mathcal{H} \to \mathcal{H}\) is the Hessian (second Fréchet
derivative).  The formula is the infinite-dimensional analogue of the finite-dimensional
Itô generator, with \(C\) playing the role of \(\sigma\sigma^\top\).&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;7-derivation-of-the-bke&quot;&gt;7. Derivation of the BKE&lt;/h2&gt;

&lt;p&gt;Since \(\{X_t\}\) is a Markov process and \(u(x,s) = \mathbb{E}[f(X_t) \mid X_s = x]\),
the process \(M_s := u(X_s, s)\) is a martingale on \([0, t]\).  Applying Itô’s formula
to \(u(X_s, s)\) in infinite dimensions and invoking the martingale property \(dM_s = 0\)
(in the drift sense) gives&lt;/p&gt;

\[\frac{\partial u}{\partial s} + \mathcal{L}u = 0.\]

&lt;p&gt;Substituting the OU drift \(b(x) = -\tfrac{1}{2}x\) yields the &lt;strong&gt;backward Kolmogorov
equation&lt;/strong&gt; for the OU process:&lt;/p&gt;

\[\boxed{-\frac{\partial u}{\partial s}(x,s)
= \left\langle -\frac{1}{2}x,\, \nabla_x u(x,s) \right\rangle_{\mathcal{H}}
+ \frac{1}{2}\operatorname{Tr}\!\left[C \cdot D^2 u(x,s)\right]}\]

&lt;p&gt;with &lt;strong&gt;terminal condition&lt;/strong&gt; \(u(x, t) = f(x)\).&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;8-structure-of-each-term&quot;&gt;8. Structure of Each Term&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Drift term&lt;/strong&gt; — \(\langle -\tfrac{1}{2}x,\, \nabla_x u \rangle_{\mathcal{H}}\)&lt;/p&gt;

&lt;p&gt;This is the directional derivative of \(u\) along the OU drift.  It accounts for the
deterministic pull of \(X_s\) toward zero: as \(s\) decreases from \(t\), the process must
travel backward through an ever-stronger restoring force.  In spectral coordinates,
this term couples each mode \(e_k\) to the gradient of \(u\) in the \(e_k\)-direction,
weighted by \(-\tfrac{1}{2}\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Diffusion term&lt;/strong&gt; — \(\tfrac{1}{2}\operatorname{Tr}[C \cdot D^2 u]\)&lt;/p&gt;

&lt;p&gt;This is the infinite-dimensional analogue of \(\tfrac{1}{2}\operatorname{Tr}[\sigma\sigma^\top H_u]\)
from finite-dimensional Itô calculus.  In the Mercer eigenbasis it reads&lt;/p&gt;

\[\tfrac{1}{2}\operatorname{Tr}[C \cdot D^2 u]
= \frac{1}{2}\sum_{k=1}^\infty \lambda_k\, \frac{\partial^2 u}{\partial e_k^2}(x,s),\]

&lt;p&gt;so the \(k\)-th mode contributes to the Laplacian-like term with weight \(\lambda_k\).
Directions with large \(\lambda_k\) (high noise energy) smooth \(u\) more aggressively;
directions with small \(\lambda_k\) contribute negligibly.  Intuitively, randomness
spreads the conditional expectation, and the rate of spreading in each direction is
governed by the corresponding eigenvalue of \(C\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Terminal condition&lt;/strong&gt; — \(u(x, t) = f(x)\)&lt;/p&gt;

&lt;p&gt;At \(s = t\) there is no residual uncertainty: \(X_t\) is already at \(x\), so the
conditional expectation collapses to \(f(x)\).  The BKE is integrated &lt;em&gt;backward&lt;/em&gt; from
this boundary condition as \(s\) decreases.&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;9-backward-vs-forward-a-comparison&quot;&gt;9. Backward vs. Forward: A Comparison&lt;/h2&gt;

&lt;p&gt;The sign \(-\partial u/\partial s\) is the hallmark of the backward direction.  In the
&lt;strong&gt;forward&lt;/strong&gt; Kolmogorov equation (Fokker-Planck), the density \(\rho(x,t)\) evolves
forward in \(t\) under the &lt;em&gt;adjoint&lt;/em&gt; generator \(\mathcal{L}^*\).  Here \(u\) evolves
backward in \(s\) under \(\mathcal{L}\) itself.&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;Equation&lt;/th&gt;
      &lt;th&gt;Quantity evolved&lt;/th&gt;
      &lt;th&gt;Operator&lt;/th&gt;
      &lt;th&gt;Time direction&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;Forward Kolmogorov (Fokker-Planck)&lt;/td&gt;
      &lt;td&gt;density \(\rho(x,t)\)&lt;/td&gt;
      &lt;td&gt;\(\mathcal{L}^*\)&lt;/td&gt;
      &lt;td&gt;forward in \(t\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Backward Kolmogorov&lt;/td&gt;
      &lt;td&gt;value \(u(x,s)\)&lt;/td&gt;
      &lt;td&gt;\(\mathcal{L}\)&lt;/td&gt;
      &lt;td&gt;backward in \(s\)&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;The two equations are dual: \(\mathcal{L}\) and \(\mathcal{L}^*\) are formal \(L^2\)-adjoints,
so the BKE and Fokker-Planck carry the same information about the process but from
complementary perspectives — one from the initial state, the other from the terminal
distribution.&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;10-connection-to-the-score-function-and-diffusion-models&quot;&gt;10. Connection to the Score Function and Diffusion Models&lt;/h2&gt;

&lt;p&gt;In the diffusion model context the terminal functional is chosen so that
\(\nabla_x u(x,s)\) recovers the &lt;strong&gt;score function&lt;/strong&gt; \(\nabla_x \log p_s(x)\), which the
denoising network learns to approximate.  Concretely, for&lt;/p&gt;

\[f(x) = \log p_t(x), \qquad p_t = \text{marginal density at time } t,\]

&lt;p&gt;the BKE governs how the score propagates &lt;em&gt;backward&lt;/em&gt; through the noising process from
\(t\) to \(s &amp;lt; t\).&lt;/p&gt;

&lt;p&gt;The spectral structure of \(C\) is directly visible in this propagation: modes with
large eigenvalue \(\lambda_k\) are noise-amplified and their score components decay
faster toward zero.  This motivates weighting the denoising loss by \(C^{-1/2}\),
yielding the &lt;strong&gt;Cameron-Martin loss&lt;/strong&gt; (whose \(\mathcal{H}_C\)-norm interpretation was
established in §3)&lt;/p&gt;

\[\mathcal{L}_{\mathrm{CM}}(\theta)
= \mathbb{E}\!\left[\left\|C^{-1/2}\bigl(\eta_\theta(X_s, s) - \eta\bigr)\right\|_{\mathcal{H}}^2\right],\]

&lt;p&gt;which down-weights high-noise directions and achieves dimension-independent
convergence rates — the key advantage of the infinite-dimensional formulation over
a naive finite-dimensional truncation.&lt;/p&gt;

&lt;h3 id=&quot;101-spectral-decoupling-transition-density&quot;&gt;10.1 Spectral Decoupling: Transition Density&lt;/h3&gt;

&lt;p&gt;By Proposition 1 in the paper (&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;eq:spectral_ou&lt;/code&gt;), projecting the OU SDE onto each
Mercer mode \(e_k\) yields an independent scalar SDE&lt;/p&gt;

\[d\alpha_k(t) = -\tfrac{1}{2}\alpha_k(t)\,dt + \sqrt{\lambda_k}\,d\beta_k(t),\]

&lt;p&gt;where \(\alpha_k(t) := \langle X_t, e_k\rangle_\mathcal{H}\) and \(\{\beta_k\}\) are
independent standard Brownian motions.  The transition density of this scalar
process is&lt;/p&gt;

\[\alpha_k(t)\mid\alpha_k(s)
\;\sim\;
\mathcal{N}\!\Bigl(e^{-(t-s)/2}\alpha_k(s),\;\lambda_k(1-e^{-(t-s)})\Bigr).\]

&lt;p&gt;&lt;strong&gt;Derivation.&lt;/strong&gt;  The scalar OU SDE is linear with constant coefficients; multiply
both sides by the integrating factor \(e^{t/2}\) and recognize the left side as an
exact differential:&lt;/p&gt;

\[d\!\left(e^{t/2}\alpha_k(t)\right) = \sqrt{\lambda_k}\,e^{t/2}\,d\beta_k(t).\]

&lt;p&gt;Integrating from \(s\) to \(t\):&lt;/p&gt;

\[e^{t/2}\alpha_k(t) - e^{s/2}\alpha_k(s)
= \sqrt{\lambda_k}\int_s^t e^{u/2}\,d\beta_k(u),\]

&lt;p&gt;so the path-wise solution is&lt;/p&gt;

\[\alpha_k(t) = e^{-(t-s)/2}\alpha_k(s)
+ \sqrt{\lambda_k}\int_s^t e^{-(t-u)/2}\,d\beta_k(u).\]

&lt;p&gt;&lt;em&gt;Conditional mean.&lt;/em&gt; The Itô integral \(\int_s^t \sqrt{\lambda_k}\,e^{-(t-u)/2}\,d\beta_k(u)\)
has zero expectation, so&lt;/p&gt;

\[\mathbb{E}[\alpha_k(t)\mid\alpha_k(s)] = e^{-(t-s)/2}\alpha_k(s).\]

&lt;p&gt;&lt;strong&gt;Why zero expectation?&lt;/strong&gt;  Three equivalent reasons:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Brownian increments have zero mean.&lt;/strong&gt; For a step-function approximation, the
Itô sum is \(\sum_i f(u_i)(\beta_k(u_{i+1})-\beta_k(u_i))\) where each increment
is independent of \(\mathcal{F}_{u_i}\) and satisfies
\(\mathbb{E}[\beta_k(u_{i+1})-\beta_k(u_i)]=0\).  The expectation of each term
is \(f(u_i)\cdot 0=0\), and the general case follows by \(L^2(\Omega)\)
approximation.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Martingale property.&lt;/strong&gt; Any Itô integral \(M_t=\int_0^t f(u)\,d\beta_k(u)\) of a
square-integrable adapted process is a martingale: \(\mathbb{E}[M_t\mid\mathcal{F}_s]=M_s\).
At \(t=s\) the integral is over the empty interval, giving \(M_s=0\), so
\(\mathbb{E}[M_t]=0\) for all \(t\geq s\).&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Deterministic integrand (Wiener integral).&lt;/strong&gt; Here \(f(u)=\sqrt{\lambda_k}e^{-(t-u)/2}\)
is deterministic, so the integral is a Wiener integral — a Gaussian with mean
\(\int_s^t f(u)\,\mathbb{E}[d\beta_k(u)] = \int_s^t f(u)\cdot 0\;du = 0\).&lt;/p&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;All three routes give the same answer; reason 3 is most direct for our setting
since the integrand is non-random.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Conditional variance.&lt;/em&gt; By the Itô isometry:&lt;/p&gt;

\[\mathrm{Var}[\alpha_k(t)\mid\alpha_k(s)]
= \lambda_k\int_s^t e^{-(t-u)}\,du
= \lambda_k\Bigl[-e^{-(t-u)}\Bigr]_s^t
= \lambda_k\bigl(1-e^{-(t-s)}\bigr).\]

&lt;p&gt;&lt;em&gt;Gaussianity.&lt;/em&gt; The stochastic integral \(\int_s^t e^{-(t-u)/2}\,d\beta_k(u)\) is a
Wiener integral (a deterministic square-integrable kernel against Brownian motion),
which is exactly Gaussian.  A Gaussian initial condition plus a Gaussian increment
is Gaussian, giving the full transition law above.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Two limiting behaviors:&lt;/strong&gt;&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;Limit&lt;/th&gt;
      &lt;th&gt;Mean&lt;/th&gt;
      &lt;th&gt;Variance&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;\(t \to s\)&lt;/td&gt;
      &lt;td&gt;\(\alpha_k(s)\)&lt;/td&gt;
      &lt;td&gt;\(0\) (recovers initial condition)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;\(t \to \infty\)&lt;/td&gt;
      &lt;td&gt;\(0\)&lt;/td&gt;
      &lt;td&gt;\(\lambda_k\) (stationary: \(\mathcal{N}(0,\lambda_k)\))&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;The \(\lambda_k\) prefactor in the variance is the key signature of colored noise:
high-energy (smooth, low-\(k\)) modes are corrupted more than high-\(k\) modes,
consistent with the Cameron-Martin picture that \(C\) weights functional directions
by their spectral energy.&lt;/p&gt;

&lt;h3 id=&quot;102-reversing-the-forward-process&quot;&gt;10.2 Reversing the Forward Process&lt;/h3&gt;

&lt;p&gt;The forward OU process corrupts a clean action \(a_0\) into pure colored noise
\(\mathcal{N}(0,C)\) as \(t \to \infty\).  Sampling a new action means running this
process in reverse: start from noise and recover a plausible \(a_0\).  The question
is what SDE the time-reversed process satisfies.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Drift-reversal formula (Anderson 1982).&lt;/strong&gt; Let \(\{X_t\}\) be a diffusion on
\(\mathcal{H}\) with forward SDE \(dX_t = b(X_t, t)\,dt + dW_t^C\).  Define the
time-reversed process \(\tilde{X}_t := X_{T-t}\).  Anderson showed that
\(\{\tilde{X}_t\}\) satisfies the reverse SDE \(d\tilde{X}_t =
\tilde{b}(\tilde{X}_t, t)\,dt + d\tilde{W}_t^C\), where \(d\tilde{W}_t^C\) is a new
\(C\)-Wiener process and the reversed drift is&lt;/p&gt;

\[\tilde{b}(x, t) = -b(x, T-t) + C\,\nabla_\mathcal{H}\log p_{T-t}(x).\]

&lt;p&gt;Reversing time flips the forward drift (\(-b\)) and adds a correction
\(C\,\nabla_\mathcal{H}\log p_{T-t}(x)\) — the score of the marginal density
premultiplied by \(C\) — that steers the reversed process toward high-density regions.
In \(\mathcal{H}\) the Euclidean gradient is replaced by the Fréchet derivative
\(\nabla_\mathcal{H}\log p_t\), and \(C\) appears because the noise covariance is \(C\),
not the identity.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Applying the formula to the OU process.&lt;/strong&gt; The OU drift is \(b(x,t) =
-\tfrac{1}{2}x\).  Substituting,&lt;/p&gt;

\[\tilde{b}(x,t)
= \tfrac{1}{2}x + C\,\nabla_\mathcal{H}\log p_t(x),\]

&lt;p&gt;giving the &lt;strong&gt;reverse SDE&lt;/strong&gt;&lt;/p&gt;

\[d\tilde{X}_t
= \Bigl[\tfrac{1}{2}\tilde{X}_t
+ C\cdot\nabla_\mathcal{H}\log p_t(\tilde{X}_t)\Bigr]dt + dW_t^C.\]

&lt;p&gt;The two drift terms have distinct roles: \(\tfrac{1}{2}\tilde{X}_t\) is the reversed
mean-reversion (now pushing &lt;em&gt;away&lt;/em&gt; from zero so the process escapes the noise
distribution), while \(C\,\nabla_\mathcal{H}\log p_t\) is the score guidance that
steers toward the data.&lt;/p&gt;

&lt;h3 id=&quot;103-the-score-function-in-spectral-coordinates&quot;&gt;10.3 The Score Function in Spectral Coordinates&lt;/h3&gt;

&lt;p&gt;The explicit marginal of the OU process is&lt;/p&gt;

\[p_t(\,\cdot \mid a_0) = \mathcal{N}\!\bigl(e^{-t/2}a_0,\; (1-e^{-t})C\bigr),\]

&lt;p&gt;so the conditional score in the Mercer eigenbasis is&lt;/p&gt;

\[\bigl[\nabla_\mathcal{H}\log p_t(x\mid a_0)\bigr]_k
= -\frac{\hat{x}_k - e^{-t/2}\widehat{(a_0)}_k}{\lambda_k(1-e^{-t})}.\]

&lt;p&gt;The \(1/\lambda_k\) factor is key: high-noise directions (large \(\lambda_k\)) have a
&lt;em&gt;gentler&lt;/em&gt; score gradient because the marginal is wider there.  Low-noise directions
(small \(\lambda_k\)) are sharp and dominate the score — the denoiser must be most
precise exactly where the noise energy is smallest.&lt;/p&gt;

&lt;p&gt;Since \(p_t\) is unknown in practice (it requires averaging over all data \(a_0\)), a
neural operator \(\eta_\theta : \mathcal{H} \times \mathbb{R}_+ \to \mathcal{H}\) is
trained via denoising score matching.  Minimizing \(\mathcal{L}_{\mathrm{CM}}(\theta)\)
above is equivalent to score matching in the \(\mathcal{H}_C\)-norm, which
automatically enforces the \(1/\lambda_k\) precision weighting.&lt;/p&gt;

&lt;h3 id=&quot;104-derivation-of-the-cameron-martin-training-loss-equation-14&quot;&gt;10.4 Derivation of the Cameron-Martin Training Loss (Equation 14)&lt;/h3&gt;

&lt;p&gt;The Cameron-Martin loss is not an ad hoc choice: it is forced by the spectral structure
of the score function established in §10.3.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Step 1: Reparametrize the forward process.&lt;/strong&gt;
The OU marginal at time \(s\) given a clean sample \(a_0\) is&lt;/p&gt;

\[X_s = e^{-s/2}a_0 + \sqrt{1-e^{-s}}\,\xi, \qquad \xi \sim \mathcal{N}(0,C).\]

&lt;p&gt;Write \(\eta := \xi\) for the injected noise.  A denoising network \(\eta_\theta(X_s, s)\)
predicts this noise from the corrupted observation.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Step 2: The plain \(\mathcal{H}\)-loss treats all modes equally.&lt;/strong&gt;
A naive squared loss in \(\mathcal{H}\) is&lt;/p&gt;

\[\mathcal{L}_{\mathrm{plain}}(\theta)
= \mathbb{E}\!\left[\|\eta_\theta(X_s,s) - \eta\|^2_{\mathcal{H}}\right]
= \mathbb{E}\!\left[\sum_{k=1}^\infty
  \bigl|\hat{\eta}_{\theta,k} - \hat{\eta}_k\bigr|^2\right].\]

&lt;p&gt;Every mode \(k\) receives the same penalty weight.  But the true noise satisfies
\(\hat{\eta}_k \sim \mathcal{N}(0, \lambda_k)\), so high-energy modes (\(\lambda_k\) large)
are noisy and hard to fit precisely; low-energy modes (\(\lambda_k\) small) are clean and
informative.  Equal weighting is therefore inconsistent with the spectral geometry of
\(\mathcal{N}(0,C)\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Step 3: The score function imposes \(1/\lambda_k\) precision requirements.&lt;/strong&gt;
From §10.3, the conditional score in mode \(k\) is&lt;/p&gt;

\[\bigl[\nabla_\mathcal{H}\log p_s(X_s \mid a_0)\bigr]_k
= -\frac{\hat{X}_{s,k} - e^{-s/2}\widehat{(a_0)}_k}{\lambda_k(1-e^{-s})}.\]

&lt;p&gt;The \(1/\lambda_k\) denominator means the score is &lt;em&gt;large&lt;/em&gt; in low-energy directions and
&lt;em&gt;small&lt;/em&gt; in high-energy directions.  A denoiser that matches high-energy modes (large
\(\lambda_k\)) precisely but errs in low-energy modes (small \(\lambda_k\)) produces a
poor score estimate exactly where it matters most for the reverse SDE.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Step 4: The Cameron-Martin norm provides the correct weighting.&lt;/strong&gt;
Define the prediction error \(\delta_k := \hat{\eta}_{\theta,k} - \hat{\eta}_k\).  The
Cameron-Martin norm of the error is&lt;/p&gt;

\[\left\|C^{-1/2}(\eta_\theta - \eta)\right\|^2_{\mathcal{H}}
= \sum_{k=1}^\infty \frac{|\delta_k|^2}{\lambda_k}.\]

&lt;p&gt;The \(1/\lambda_k\) weight is the inverse of the per-mode noise variance.  In Bayesian
terms, this is the &lt;em&gt;precision&lt;/em&gt; (reciprocal variance) of \(\hat{\eta}_k\).  Penalizing by
\(1/\lambda_k\) means:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Low-energy modes (small \(\lambda_k\)): errors penalized &lt;em&gt;heavily&lt;/em&gt; — the denoiser must
be precise where the score is sharp.&lt;/li&gt;
  &lt;li&gt;High-energy modes (large \(\lambda_k\)): errors penalized &lt;em&gt;mildly&lt;/em&gt; — the score is
gentle there and imprecision matters less.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Step 5: Equivalence with spectral score matching.&lt;/strong&gt;
The score \(\nabla_\mathcal{H}\log p_s\) is a gradient — a &lt;em&gt;dual&lt;/em&gt; object in \(\mathcal{H}_C^*\),
not a function in \(\mathcal{H}_C\).  The dual norm to the CM norm \(\|C^{-1/2}\cdot\|_\mathcal{H}\)
(which penalizes functions by \(1/\lambda_k\)) is \(\|C^{1/2}\cdot\|_\mathcal{H}\)
(which penalizes dual objects by \(\lambda_k\)).  Therefore the score-matching objective
measured in the dual-CM metric is&lt;/p&gt;

\[\mathcal{L}_{\mathrm{SM-CM}}(\theta)
= \mathbb{E}\!\left[\left\|
  C^{1/2}\bigl(\widehat{\nabla\log p}_{\theta,s} - \nabla_\mathcal{H}\log p_s\bigr)
\right\|^2_{\mathcal{H}}\right].\]

&lt;p&gt;&lt;em&gt;(Using \(C^{-1/2}\) on the score would give \(1/\lambda_k^2\) weighting — over-penalized and
inconsistent with the CM geometry.  Using \(C^{1/2}\) gives \(\lambda_k \cdot 1/\lambda_k^2 = 1/\lambda_k\)
per mode, matching the CM noise loss.)&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;To see this explicitly, parametrize the network score as
\(\hat{s}_{\theta,k} = -\hat{\eta}_{\theta,k}/(\lambda_k\sqrt{1-e^{-s}})\), so the
mode-\(k\) score error is&lt;/p&gt;

\[\hat{s}_{\theta,k} - [\nabla\log p_s]_k
= -\frac{\hat{\eta}_{\theta,k} - \hat{\eta}_k}{\lambda_k\sqrt{1-e^{-s}}}.\]

&lt;p&gt;Then&lt;/p&gt;

\[\left\|C^{1/2}(s_\theta - \nabla\log p_s)\right\|^2_\mathcal{H}
= \sum_k \lambda_k\,\bigl|\hat{s}_{\theta,k} - [\nabla\log p_s]_k\bigr|^2
= \frac{1}{1-e^{-s}}\sum_k \frac{|\hat{\eta}_{\theta,k}-\hat{\eta}_k|^2}{\lambda_k}
= \frac{1}{1-e^{-s}}\left\|C^{-1/2}(\eta_\theta-\eta)\right\|^2_\mathcal{H}.\]

&lt;p&gt;Taking expectations over \((X_s, s)\) and absorbing the time-averaging factor
\(\mathbb{E}_s[1/(1-e^{-s})]\) into the schedule yields&lt;/p&gt;

\[\mathbb{E}\!\left[\left\|C^{-1/2}\bigl(\eta_\theta(X_s,s) - \eta\bigr)\right\|^2_{\mathcal{H}}\right]
= \mathbb{E}\!\left[\sum_{k=1}^\infty \frac{|\hat{\eta}_{\theta,k} - \hat{\eta}_k|^2}{\lambda_k}\right],\]

&lt;p&gt;which is exactly the Cameron-Martin loss&lt;/p&gt;

\[\boxed{\mathcal{L}_{\mathrm{CM}}(\theta)
= \mathbb{E}\!\left[\left\|C^{-1/2}\bigl(\eta_\theta(X_s,s) - \eta\bigr)\right\|^2_{\mathcal{H}}\right].}\]

&lt;p&gt;&lt;strong&gt;Why not use \(C^{-1}\) or \(C^{-2}\)?&lt;/strong&gt; The operator \(C^{-1/2}\) arises because the
Cameron-Martin space inner product is \(\langle h_1, h_2\rangle_{\mathcal{H}_C}
= \langle C^{-1/2}h_1, C^{-1/2}h_2\rangle_{\mathcal{H}}\) (§3).  The loss is the
squared \(\mathcal{H}_C\)-norm of the prediction error: a single application of
\(C^{-1/2}\) maps the error from \(\mathcal{H}\) into the Cameron-Martin space where the
Gaussian measure \(\mathcal{N}(0,C)\) has unit variance in every direction.  Using
\(C^{-1}\) would over-penalize (\(1/\lambda_k^2\) weighting) and correspond to matching
the score in the \(\mathcal{H}\)-norm rather than the \(\mathcal{H}_C\)-norm.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Summary.&lt;/strong&gt; The CM loss is the unique denoising objective consistent with the
Cameron-Martin geometry of \(\mathcal{N}(0,C)\).  It arises from three simultaneous
requirements: (i) match the score function, (ii) respect the \(1/\lambda_k\)
precision structure imposed by the score’s spectral form, and (iii) measure errors
in the inner product of the Cameron-Martin space \(\mathcal{H}_C\).  Any other
\(\mathcal{H}\)-norm would violate at least one of these.&lt;/p&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;105-sampling-algorithm-and-why-colored-noise-is-required&quot;&gt;10.5 Sampling Algorithm and Why Colored Noise Is Required&lt;/h3&gt;

&lt;p&gt;Sampling discretizes the reverse SDE from \(t = T\) to \(t = 0\):&lt;/p&gt;

\[X_{t - \Delta t}
= \mu_\theta(X_t, t) + \sqrt{\tilde{\beta}_t}\,\zeta_t,
\qquad \zeta_t \sim \mathcal{N}(0, C),\]

&lt;p&gt;where \(\mu_\theta\) is the posterior mean from \(\eta_\theta\) and \(\tilde{\beta}_t\) is
the posterior variance schedule.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why \(\zeta_t \sim \mathcal{N}(0,C)\) (colored) and not \(\mathcal{N}(0, I)\) (white)?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The forward noising injects \(C\)-colored increments \(dW_t^C\) at every step; the
reverse SDE is also driven by \(dW_t^C\).  Using white noise \(\mathcal{N}(0,I)\)
instead would inject energy &lt;em&gt;outside&lt;/em&gt; the Cameron-Martin space
\(E = C^{1/2}(\mathcal{H})\).  By the Cameron-Martin theorem (§3), any perturbation
outside \(E\) produces a measure mutually singular with \(\mathcal{N}(0,C)\), meaning
the reverse trajectory immediately leaves the support of the data distribution.&lt;/p&gt;

&lt;p&gt;In spectral terms: white noise injects equal energy in every mode \(e_k\); the data
distribution concentrates on low-\(k\) modes (large \(\lambda_k\)); colored noise
\(\mathcal{N}(0,C)\) injects energy proportional to \(\lambda_k\), keeping the
trajectory inside \(E\) at every step.  The replacement
\(\mathcal{N}(0,I) \to \mathcal{N}(0,C)\) is therefore not cosmetic — it is a
measure-theoretic necessity.&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;11-advantage-of-working-with-the-kolmogorov-chapman-equation&quot;&gt;11. Advantage of Working with the Kolmogorov-Chapman Equation&lt;/h2&gt;

&lt;p&gt;This is the central question motivating the paper.  Here is a precise comparison.&lt;/p&gt;

&lt;p&gt;The OU SDE describes the forward dynamics of a single stochastic path:&lt;/p&gt;

\[dX_t = -\frac{1}{2}X_t\, dt + dW_t^C.\]

&lt;p&gt;It tells you how noise is injected into the process.  To get any expectation from it,
you must simulate many trajectories and average — a Monte Carlo procedure that is
expensive, stochastic, and gives no analytic handle on what the denoiser should learn.&lt;/p&gt;

&lt;p&gt;The BKE is a deterministic PDE for the value function \(u(x,s) = \mathbb{E}[f(X_t) \mid X_s = x]\):&lt;/p&gt;

\[-\frac{\partial u}{\partial s}
= \left\langle -\frac{1}{2}x,\, \nabla_x u \right\rangle_{\mathcal{H}}
+ \frac{1}{2}\operatorname{Tr}[C \cdot D^2 u].\]

&lt;p&gt;It tells you what the conditional expectation satisfies — a structured, smooth,
deterministic object.  The advantages are concrete:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Stochastic → deterministic.&lt;/strong&gt;
The OU SDE is a random process; the BKE is a PDE.  Everything you want to learn about
the denoising distribution — the score \(\nabla_x \log p_s(x)\), the transition density,
the conditional mean — lives in \(u\) and is governed by a deterministic equation.
Score matching becomes structured regression against a PDE solution, not averaging
over Monte Carlo paths.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Online physics-aware diagnostic (Kolmogorov residual).&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Derivation.&lt;/strong&gt;  The BKE states that the true value function \(u\) satisfies&lt;/p&gt;

\[-\frac{\partial u}{\partial s}
= \left\langle -\frac{1}{2}x,\, \nabla_x u \right\rangle_{\mathcal{H}}
+ \frac{1}{2}\operatorname{Tr}[C \cdot D^2 u].\]

&lt;p&gt;Move every term to the left-hand side (add \(\partial_s u\) to both sides, subtract the
right-hand side from both sides):&lt;/p&gt;

\[\frac{\partial u}{\partial s}
+ \left\langle -\frac{1}{2}x,\, \nabla_x u \right\rangle_{\mathcal{H}}
+ \frac{1}{2}\operatorname{Tr}[C \cdot D^2 u] = 0.\]

&lt;p&gt;This identity holds &lt;em&gt;exactly&lt;/em&gt; for the true solution.  Substituting the learned
approximation \(u_\theta\) in place of \(u\) and measuring how much the identity is
violated defines the &lt;strong&gt;Kolmogorov residual&lt;/strong&gt;:&lt;/p&gt;

\[\mathcal{R}_\theta(x,s)
:= \frac{\partial u_\theta}{\partial s}
+ \left\langle -\frac{1}{2}x,\, \nabla_x u_\theta \right\rangle_{\mathcal{H}}
+ \frac{1}{2}\operatorname{Tr}[C \cdot D^2 u_\theta].\]

&lt;p&gt;By construction \(\mathcal{R}_\theta \equiv 0\) if and only if \(u_\theta\) solves the
BKE exactly.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;“Pointwise PDE error, not a statistical one.”&lt;/strong&gt;
A statistical error requires averaging over samples — it is an expectation.
\(\mathcal{R}_\theta(x,s)\) is a deterministic quantity: plug a single \((x,s)\) into
\(u_\theta\) and its derivatives and compute a number.  No expectation, no law of large
numbers, no variance.  It is &lt;em&gt;pointwise&lt;/em&gt; because it lives at a specific \((x,s)\), and
it is a &lt;em&gt;PDE error&lt;/em&gt; because it measures violation of a differential equation, not an
empirical risk.  Contrast with the training loss
\(\mathcal{L}_\text{CM}(\theta) = \mathbb{E}[\cdots]\), which is genuinely
statistical — estimated over a batch.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;“Evaluate at inference without environment interaction.”&lt;/strong&gt;
The three quantities in \(\mathcal{R}_\theta(x,s)\) are:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;\(\partial_s u_\theta\) — a derivative of the network w.r.t. its time input,&lt;/li&gt;
  &lt;li&gt;\(\langle -\tfrac{1}{2}x,\,\nabla_x u_\theta\rangle\) — the directional derivative of the network w.r.t. its state input,&lt;/li&gt;
  &lt;li&gt;\(\tfrac{1}{2}\operatorname{Tr}[C \cdot D^2 u_\theta]\) — a weighted trace of the Hessian.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;All three come from automatic differentiation applied to \(u_\theta\).  You need only
the current query point \((x,s)\) and the model weights — no policy rollout, no new
data collection, no task outcome.  If only the OU SDE were available, there is no
algebraic identity to check; a path either converges or it does not, and measuring
that requires running trajectories.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Spectral decoupling at the function level.&lt;/strong&gt;
In the Mercer eigenbasis \(\{e_k\}\) of \(C\), the diffusion term becomes&lt;/p&gt;

\[\frac{1}{2}\operatorname{Tr}[C \cdot D^2 u]
= \frac{1}{2}\sum_{k=1}^\infty \lambda_k\, \frac{\partial^2 u}{\partial e_k^2}.\]

&lt;p&gt;Each mode contributes independently, weighted by \(\lambda_k\).  This produces the
emergent two-scale hierarchy (low-\(k\) locomotion modes vs. high-\(k\) manipulation modes)
as a consequence of the BKE structure, not a design choice.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. Dimension-independent convergence.&lt;/strong&gt;
The BKE shows that the Cameron-Martin loss achieves a total-variation convergence bound
depending only on \(r_{\mathrm{eff}}(C) = (\sum_k\lambda_k)^2/(\sum_k\lambda_k^2)\),
not on the discretization dimension \(d\).  This is provable through the BKE’s PDE
structure; from the SDE alone, the analysis sees \(d\) explicitly and degrades as
\(d \to \infty\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;5. Inversion is well-posed.&lt;/strong&gt;
The paper frames policy recovery as an inverse problem.  The BKE casts this as PDE
inversion — recovering \(u\) from observations of \(f(X_t)\) — which is better posed and
admits regularization through \(\mathcal{L}\).  Inverting the SDE directly (recovering
the drift from paths) is a much harder stochastic inverse problem with no clean PDE
structure to exploit.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Summary:&lt;/em&gt; the OU SDE describes the noise process; the BKE describes the function
the denoiser must approximate — and it does so deterministically, spectrally, and in
a way that is directly checkable at inference time.&lt;/p&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;12-theorem-1-dimension-independent-convergence-complete-proof&quot;&gt;12. THEOREM 1: Dimension-Independent Convergence (Complete Proof)&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Verification Status:&lt;/strong&gt; Proven across 5 turns with full rigor. Each phase below references the complete derivations above.&lt;/p&gt;

&lt;h3 id=&quot;121-theorem-statement&quot;&gt;12.1 Theorem Statement&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;Theorem 1 (Dimension-Independent Convergence).&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Let \(\mu_{\text{data}}\) be a probability measure on \(\mathcal{H} = L^2([0,T], \mathbb{R}^{d_a})\) with finite second moment and full support. Let \(\mu_\theta\) denote the distribution of trajectories generated by the infinite-dimensional diffusion policy trained with Cameron-Martin loss
\(\mathcal{L}_{\mathrm{CM}}(\theta) = \mathbb{E}_{s,X_s,\eta}\left[\left\|\mathcal{C}_\mu^{-1/2}(\eta_\theta(X_s,s)-\eta)\right\|_\mathcal{H}^2\right],\)
where \(\eta \sim \mathcal{N}(0,\mathcal{C}_\mu)\).&lt;/p&gt;

&lt;p&gt;Then the total variation distance satisfies
\(\|\mu_\theta - \mu_{\text{data}}\|_{\text{TV}} \le C_1\sqrt{\mathcal{L}_{\mathrm{CM}}(\theta)} + C_2 e^{-T/2},\)
where the constants \(C_1, C_2 &amp;gt; 0\) depend on \(\operatorname{Tr}(\mathcal{C}_\mu)\) but are &lt;strong&gt;independent of:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;The discretization resolution (we use no discretization)&lt;/li&gt;
  &lt;li&gt;The planning horizon \(T\) (except exponentially)&lt;/li&gt;
  &lt;li&gt;The action dimension \(d_a\)&lt;/li&gt;
&lt;/ul&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;122-proof-structure-seven-phases&quot;&gt;12.2 Proof Structure: Seven Phases&lt;/h3&gt;

&lt;p&gt;The proof assembles measure theory (§1–3), dynamical systems (§5–7), and information theory (Girsanov, Pinsker) to achieve dimension-independence.&lt;/p&gt;

&lt;hr /&gt;

&lt;h4 id=&quot;phase-1-measure-theoretic-foundation-13&quot;&gt;&lt;strong&gt;Phase 1: Measure-Theoretic Foundation (§1–3)&lt;/strong&gt;&lt;/h4&gt;

&lt;p&gt;&lt;strong&gt;Cameron-Martin Space Setting&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The space \(\mathcal{H} = L^2([0,T], \mathbb{R}^{d_a})\) admits no Lebesgue reference measure. Denoising operates in the Cameron-Martin space:
\(\mathcal{H}_\mathcal{C} = \left\{h \in \mathcal{H} : \|h\|_{\mathcal{H}_\mathcal{C}}^2 := \sum_{k=1}^\infty \frac{|\hat{h}_k|^2}{\lambda_k} &amp;lt; \infty\right\},\)
where \(\hat{h}_k = \langle h, e_k \rangle_\mathcal{H}\) and \(\{(\lambda_k, e_k)\}\) are Mercer eigenpairs of \(\mathcal{C}_\mu\) (§1).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Trace-Class Condition&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;By Mercer’s theorem (§1), the covariance operator is trace-class:
\(\operatorname{Tr}(\mathcal{C}_\mu) = \sum_{k=1}^\infty \lambda_k &amp;lt; \infty.\)&lt;/p&gt;

&lt;p&gt;For the Matérn kernel with smoothness parameter \(\nu = 3/2\), the eigenvalues decay as \(\lambda_k \sim k^{-3}\), ensuring fast convergence.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Absolute Continuity&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;By the Cameron-Martin theorem (§3), the Gaussian measure \(\mu_0 = \mathcal{N}(0,\mathcal{C}_\mu)\) is absolutely continuous under translation only within \(\mathcal{H}_\mathcal{C}\). This is the fundamental constraint that forces the use of the Cameron-Martin norm in the loss.&lt;/p&gt;

&lt;hr /&gt;

&lt;h4 id=&quot;phase-2-forward-process--backward-kolmogorov-characterization-47&quot;&gt;&lt;strong&gt;Phase 2: Forward Process &amp;amp; Backward Kolmogorov Characterization (§4–7)&lt;/strong&gt;&lt;/h4&gt;

&lt;p&gt;&lt;strong&gt;OU Forward Process&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The forward diffusion is (§5):
\(dX_s = -\tfrac{1}{2}X_s\, ds + dW_s^{\mathcal{C}_\mu}, \quad X_0 \sim \mu_{\text{data}}.\)&lt;/p&gt;

&lt;p&gt;The \(\mathcal{C}_\mu\)-Wiener process (§4) is:
\(W_s^{\mathcal{C}_\mu} = \sum_{k=1}^\infty \sqrt{\lambda_k}\, \omega_k(s)\, e_k,\)
which converges in \(L^2(\Omega; \mathcal{H})\) by the trace-class condition.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Conditional Distribution (Exact)&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;For \(0 \le s \le t\):
\(X_s \mid X_0 = a_0 \sim \mathcal{N}(e^{-s/2}a_0, (1-e^{-s})\mathcal{C}_\mu).\)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Value Function &amp;amp; Generator&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Define \(u(x,s) := \mathbb{E}[f(X_t) | X_s = x]\) (§5). The infinitesimal generator (§6) is:
\(\mathcal{L}u(x) = \left\langle -\tfrac{1}{2}x, \nabla_x u(x) \right\rangle_\mathcal{H} + \tfrac{1}{2}\operatorname{Tr}[\mathcal{C}_\mu \cdot \nabla_x^2 u(x)].\)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Backward Kolmogorov Equation&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;By Itô’s formula and the martingale property (§7):
\(\boxed{-\frac{\partial u}{\partial s}(x,s) = \left\langle -\tfrac{1}{2}x,\nabla_x u(x,s)\right\rangle_\mathcal{H} + \tfrac{1}{2}\operatorname{Tr}[\mathcal{C}_\mu \cdot \nabla_x^2 u(x,s)]}\)
with terminal condition \(u(x,t) = f(x)\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Key Fact:&lt;/strong&gt; This PDE characterization &lt;strong&gt;avoids Lebesgue densities entirely&lt;/strong&gt;. No \(\nabla \log p\) is computed directly; instead, the score is recovered as
\(\nabla_x \log p_s(x) = \mathcal{C}_\mu^{-1} \nabla_x u(x,s).\)&lt;/p&gt;

&lt;hr /&gt;

&lt;h4 id=&quot;phase-3-error-decomposition-via-girsanov--kl-divergence&quot;&gt;&lt;strong&gt;Phase 3: Error Decomposition via Girsanov &amp;amp; KL Divergence&lt;/strong&gt;&lt;/h4&gt;

&lt;p&gt;&lt;strong&gt;Two Diffusions&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Both \(\mu_{\text{data}}\) and \(\mu_\theta\) are generated by diffusions:
\(dX_s^{\text{data}} = -\tfrac{1}{2}X_s\,ds + \eta^*(X_s,s)\,ds + dW_s^{\mathcal{C}_\mu},\)
\(dX_s^\theta = -\tfrac{1}{2}X_s\,ds + \eta_\theta(X_s,s)\,ds + dW_s^{\mathcal{C}_\mu},\)
where \(\eta^*\) is the optimal denoising direction and \(\eta_\theta\) is the learned denoiser.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Girsanov’s Theorem&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;By Girsanov (infinite-dimensional version), the Radon-Nikodym derivative is:
\(\frac{d\mu_\theta}{d\mu_{\text{data}}}(X^{\text{data}}) = \exp\left(\int_0^T \langle \eta_\theta - \eta^*, dW_s^{\mathcal{C}_\mu}\rangle_\mathcal{H} - \tfrac{1}{2}\int_0^T \|\eta_\theta-\eta^*\|_{\mathcal{H}_\mathcal{C}}^2\,ds\right).\)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;KL Divergence&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The stochastic integral term vanishes in expectation (martingale property + Itô isometry for trace-class noise):
\(\operatorname{KL}(\mu_{\text{data}} \| \mu_\theta) = \tfrac{1}{2}\int_0^T \mathbb{E}_s\left[\|\eta_\theta(X_s,s) - \eta^*(X_s,s)\|_{\mathcal{H}_\mathcal{C}}^2\right] ds.\)&lt;/p&gt;

&lt;p&gt;Define the score error \(\varepsilon_s(x) := \eta_\theta(x,s) - \eta^*(x,s)\):
\(\operatorname{KL}(\mu_{\text{data}}\|\mu_\theta) = \tfrac{1}{2}\int_0^T \mathbb{E}_s\left[\|\varepsilon_s(X_s)\|_{\mathcal{H}_\mathcal{C}}^2\right] ds.\)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Key Fact:&lt;/strong&gt; The Girsanov constant (1/2) is &lt;strong&gt;universal&lt;/strong&gt;. The norm \(\|\cdot\|_{\mathcal{H}_\mathcal{C}}\) operates on function space, not discretized vectors—&lt;strong&gt;no dimension dependence&lt;/strong&gt;.&lt;/p&gt;

&lt;hr /&gt;

&lt;h4 id=&quot;phase-4-cameron-martin-loss-controls-score-error&quot;&gt;&lt;strong&gt;Phase 4: Cameron-Martin Loss Controls Score Error&lt;/strong&gt;&lt;/h4&gt;

&lt;p&gt;&lt;strong&gt;Loss Definition&lt;/strong&gt;&lt;/p&gt;

\[\mathcal{L}_{\mathrm{CM}}(\theta) = \mathbb{E}_{s,X_s,\eta \sim \mathcal{N}(0,\mathcal{C}_\mu)}\left[\left\|\mathcal{C}_\mu^{-1/2}(\eta_\theta(X_s,s)-\eta)\right\|_\mathcal{H}^2\right].\]

&lt;p&gt;&lt;strong&gt;Norm Equivalence&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;By definition of the Cameron-Martin norm (§3):
\(\left\|\mathcal{C}_\mu^{-1/2}\varepsilon_s\right\|_\mathcal{H} = \|\varepsilon_s\|_{\mathcal{H}_\mathcal{C}}.\)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Loss-Error Relationship&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Expand the loss in terms of error \(\varepsilon_s\). By orthogonality of residuals at optimality:
\(\mathcal{L}_{\mathrm{CM}}(\theta) = \text{const} + \mathbb{E}_{s,X_s}\left[\|\varepsilon_s(X_s)\|_{\mathcal{H}_\mathcal{C}}^2\right].\)&lt;/p&gt;

&lt;p&gt;Integrating over time:
\(\int_0^T \mathbb{E}_s\left[\|\varepsilon_s(X_s)\|_{\mathcal{H}_\mathcal{C}}^2\right] ds \lesssim \mathcal{L}_{\mathrm{CM}}(\theta).\)&lt;/p&gt;

&lt;p&gt;This is the &lt;strong&gt;precise derivation given in §10.4&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Key Fact:&lt;/strong&gt; The loss &lt;strong&gt;directly bounds score error in the Cameron-Martin norm&lt;/strong&gt;. No dimensional scaling factors appear.&lt;/p&gt;

&lt;hr /&gt;

&lt;h4 id=&quot;phase-5-kl-to-total-variation-via-pinskers-inequality&quot;&gt;&lt;strong&gt;Phase 5: KL to Total Variation via Pinsker’s Inequality&lt;/strong&gt;&lt;/h4&gt;

&lt;p&gt;&lt;strong&gt;Pinsker’s Inequality&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;For any two probability measures on a separable metric space:
\(\|\mu_\theta - \mu_{\text{data}}\|_{\text{TV}} \le \sqrt{\frac{1}{2}\operatorname{KL}(\mu_{\text{data}} \| \mu_\theta)}.\)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Combining Phases 3 &amp;amp; 4&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;From Phase 3:
\(\operatorname{KL}(\mu_{\text{data}} \| \mu_\theta) = \tfrac{1}{2}\int_0^T \mathbb{E}_s\left[\|\varepsilon_s(X_s)\|_{\mathcal{H}_\mathcal{C}}^2\right] ds.\)&lt;/p&gt;

&lt;p&gt;From Phase 4:
\(\int_0^T \mathbb{E}_s\left[\|\varepsilon_s(X_s)\|_{\mathcal{H}_\mathcal{C}}^2\right] ds \lesssim \mathcal{L}_{\mathrm{CM}}(\theta).\)&lt;/p&gt;

&lt;p&gt;Therefore:
\(\operatorname{KL}(\mu_{\text{data}} \| \mu_\theta) \lesssim \mathcal{L}_{\mathrm{CM}}(\theta).\)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Final Bound&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;By Pinsker:
\(\|\mu_\theta - \mu_{\text{data}}\|_{\text{TV}} \le \sqrt{\tfrac{1}{2} C&apos; \mathcal{L}_{\mathrm{CM}}(\theta)} = C_1\sqrt{\mathcal{L}_{\mathrm{CM}}(\theta)},\)
where \(C_1 = \sqrt{C&apos;/2}\) is determined by Girsanov and Pinsker constants.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;\(C_1\) Dependency:&lt;/strong&gt;
\(C_1 \text{ depends only on } \operatorname{Tr}(\mathcal{C}_\mu), \text{ universal constants}.\)&lt;/p&gt;

&lt;p&gt;\(C_1\) is &lt;strong&gt;independent of \(d_a\), discretization \(N\), and planning horizon \(T\)&lt;/strong&gt;.&lt;/p&gt;

&lt;hr /&gt;

&lt;h4 id=&quot;phase-6-bke-approximation-error&quot;&gt;&lt;strong&gt;Phase 6: BKE Approximation Error&lt;/strong&gt;&lt;/h4&gt;

&lt;p&gt;&lt;strong&gt;Picard Iteration&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The BKE is solved numerically via Picard iteration from \(s = t\) back to \(s = 0\) (§7):
\(u_n(x,s) = f(x) - \int_s^t \mathcal{L}u_{n-1}(X_\tau,\tau)\,d\tau.\)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;OU Semigroup Contraction&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The OU operator generates a &lt;strong&gt;contractive semigroup&lt;/strong&gt; (§10.2, Spectral Decoupling):
\(\|P_\tau u\|_\mathcal{H} \le e^{-\tau/2}\|u\|_\mathcal{H}.\)&lt;/p&gt;

&lt;p&gt;This follows from the spectral decomposition: eigenvalues of the OU operator are \(-\lambda_k/2\) for Mercer eigenvalues \(\lambda_k &amp;gt; 0\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Iteration Error Bound&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Define error \(e_n := u^* - u_n\). Recursively:
\(\|e_n(x,0)\|_\mathcal{H} = \left\|\int_0^t \mathcal{L}e_{n-1}(\tau)\,d\tau\right\|_\mathcal{H} \le \int_0^t e^{-\tau/2}\|e_{n-1}\|_\mathcal{H}\,d\tau.\)&lt;/p&gt;

&lt;p&gt;For large \(t\), \(\int_0^t e^{-\tau/2}\,d\tau = 2(1-e^{-t/2}) \le 1\). Thus:
\(\|e_n(x,0)\|_\mathcal{H} \le \left(\tfrac{1}{2}\right)^n\|f\|_\infty.\)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Tail Error&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;After sufficient iterations, the residual is dominated by:
\(\left\|u_{\text{numerical}}(x,0) - u^*(x,0)\right\|_\mathcal{H} \le C_2 e^{-t/2},\)
where \(C_2\) depends on \(\|f\|_\infty\) only.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why \(e^{-T/2}\)?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The exponential decay arises from the time-integral of the OU contraction:
\(\int_0^T e^{-\tau/2}\,d\tau = 2(1-e^{-T/2}).\)
For large \(T\), this approaches 2 exponentially. The residual after many iterations is \(O(e^{-T/2})\).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Key Fact:&lt;/strong&gt; This exponential decay is intrinsic to OU dynamics, &lt;strong&gt;independent of discretization or action dimension&lt;/strong&gt;.&lt;/p&gt;

&lt;hr /&gt;

&lt;h4 id=&quot;phase-7-final-bound-assembly&quot;&gt;&lt;strong&gt;Phase 7: Final Bound Assembly&lt;/strong&gt;&lt;/h4&gt;

&lt;p&gt;&lt;strong&gt;Combining All Phases&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Learning Error (Phase 5):&lt;/strong&gt;
\(\|\mu_\theta - \mu_{\text{data}}\|_{\text{TV}} \le C_1\sqrt{\mathcal{L}_{\mathrm{CM}}(\theta)}.\)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Approximation Error (Phase 6):&lt;/strong&gt;
\(\text{BKE Tail Error} \le C_2 e^{-T/2}.\)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Final Theorem&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Adding the approximation error (which accounts for numerical integration):
\(\boxed{\|\mu_\theta - \mu_{\text{data}}\|_{\text{TV}} \le C_1\sqrt{\mathcal{L}_{\mathrm{CM}}(\theta)} + C_2 e^{-T/2}}\)&lt;/p&gt;

&lt;p&gt;where:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;\(C_1 &amp;gt; 0\) depends only on: \(\operatorname{Tr}(\mathcal{C}_\mu)\), Pinsker constant, Girsanov constant&lt;/li&gt;
  &lt;li&gt;\(C_2 &amp;gt; 0\) depends only on: \(\|f\|_\infty\), OU operator properties&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Neither depends on:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;Action dimension \(d_a\)&lt;/li&gt;
  &lt;li&gt;Discretization resolution \(N\)&lt;/li&gt;
  &lt;li&gt;Planning horizon \(T\) (except exponentially)&lt;/li&gt;
&lt;/ul&gt;

\[\text{Q.E.D.}\]

&lt;hr /&gt;

&lt;h3 id=&quot;123-why-dimension-independence-holds-mechanism&quot;&gt;12.3 Why Dimension-Independence Holds: Mechanism&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;Finite-Dimensional DDPM (Degradation)&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Action space: \(\mathbb{R}^d\)&lt;/li&gt;
  &lt;li&gt;Discretization: Grid \(\mathbb{R}^N\) with \(N \gg d\)&lt;/li&gt;
  &lt;li&gt;Noise: \(\eta \sim \mathcal{N}(0, \sigma^2 I_N)\)&lt;/li&gt;
  &lt;li&gt;Loss: \(\mathbb{E}[\|\eta_\theta - \eta\|_N^2]\) sums over \(N\) independent components&lt;/li&gt;
  &lt;li&gt;Convergence theorem: \(\|\mu_\theta - \mu_{\text{data}}\|_{\text{TV}} = O(\sqrt{N} \sqrt{\text{loss}})\)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Dimensional explosion:&lt;/strong&gt; As \(N \to \infty\), convergence degrades as \(O(\sqrt{N})\).&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;&lt;strong&gt;Our Infinite-Dimensional Formulation (No Degradation)&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Action space: \(\mathcal{H} = L^2([0,T], \mathbb{R}^{d_a})\)&lt;/li&gt;
  &lt;li&gt;Discretization: &lt;strong&gt;None&lt;/strong&gt; (infinite-dimensional from the start)&lt;/li&gt;
  &lt;li&gt;Noise: \(\eta = \mathcal{C}_\mu^{1/2}\xi \sim \mathcal{N}(0,\mathcal{C}_\mu)\) (spectrally structured)&lt;/li&gt;
  &lt;li&gt;Loss: \(\mathcal{L}_{\mathrm{CM}} = \mathbb{E}[\|\mathcal{C}_\mu^{-1/2}(\eta_\theta - \eta)\|_\mathcal{H}^2]\) (covariance-weighted)&lt;/li&gt;
  &lt;li&gt;Convergence: \(\|\mu_\theta - \mu_{\text{data}}\|_{\text{TV}} = O(\sqrt{\text{loss}})\)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;No dimensional factor:&lt;/strong&gt; Constants depend only on \(\operatorname{Tr}(\mathcal{C}_\mu)\) (a &lt;strong&gt;fixed scalar property&lt;/strong&gt; of the data), not on \(d_a\) or discretization.&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;&lt;strong&gt;The Key Insight: Cameron-Martin Weighting&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The loss is &lt;strong&gt;not&lt;/strong&gt;:
\(\mathcal{L}_{\text{naive}} = \mathbb{E}\left[\|\eta_\theta - \eta\|_\mathcal{H}^2\right].\)&lt;/p&gt;

&lt;p&gt;Instead it is:
\(\mathcal{L}_{\mathrm{CM}} = \mathbb{E}\left[\|\mathcal{C}_\mu^{-1/2}(\eta_\theta - \eta)\|_\mathcal{H}^2\right] = \mathbb{E}\left[\sum_{k=1}^\infty \frac{|\eta_{\theta,k} - \eta_k|^2}{\lambda_k}\right].\)&lt;/p&gt;

&lt;p&gt;The inverse covariance weighting \(\mathcal{C}_\mu^{-1/2}\) &lt;strong&gt;automatically down-weights high-variance directions&lt;/strong&gt;, compensating for the infinite-dimensional geometry.&lt;/p&gt;

&lt;p&gt;In discrete settings, \(\mathcal{C}_\mu^{-1/2}\) would induce a scaling factor \(O(1/N)\). But since we work directly in function space without discretization, this scaling never occurs.&lt;/p&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;124-dimension-independence-dependency-table&quot;&gt;12.4 Dimension-Independence Dependency Table&lt;/h3&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;Component&lt;/th&gt;
      &lt;th&gt;Depends On&lt;/th&gt;
      &lt;th&gt;Independent Of&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;\(\mathcal{H} = L^2([0,T], \mathbb{R}^{d_a})\)&lt;/td&gt;
      &lt;td&gt;Time \(T\), action dim \(d_a\)&lt;/td&gt;
      &lt;td&gt;Discretization \(N\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Matérn kernel \(k(t,s)\)&lt;/td&gt;
      &lt;td&gt;Length scale \(\ell\), smoothness&lt;/td&gt;
      &lt;td&gt;\(d_a\) (temporal kernel)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;\(\operatorname{Tr}(\mathcal{C}_\mu)\)&lt;/td&gt;
      &lt;td&gt;Spectral sum&lt;/td&gt;
      &lt;td&gt;\(d_a\) (scalar eigenvalue sum)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Cameron-Martin space \(\mathcal{H}_\mathcal{C}\)&lt;/td&gt;
      &lt;td&gt;Covariance \(\mathcal{C}_\mu\)&lt;/td&gt;
      &lt;td&gt;\(d_a\) (RKHS depends on covariance)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Girsanov constant&lt;/td&gt;
      &lt;td&gt;Universal (1/2)&lt;/td&gt;
      &lt;td&gt;Everything&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Pinsker constant&lt;/td&gt;
      &lt;td&gt;Universal (\(\sqrt{1/2}\))&lt;/td&gt;
      &lt;td&gt;Everything&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;OU contraction \(e^{-\tau/2}\)&lt;/td&gt;
      &lt;td&gt;Time \(\tau\), operator spectrum&lt;/td&gt;
      &lt;td&gt;\(d_a\), \(N\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;\(C_1\) coefficient&lt;/td&gt;
      &lt;td&gt;\(\operatorname{Tr}(\mathcal{C}_\mu)\)&lt;/td&gt;
      &lt;td&gt;\(d_a\), \(N\), \(T\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;\(C_2\) coefficient&lt;/td&gt;
      &lt;td&gt;OU operator properties&lt;/td&gt;
      &lt;td&gt;\(d_a\), \(N\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Cameron-Martin loss \(\mathcal{L}_{\mathrm{CM}}(\theta)\)&lt;/td&gt;
      &lt;td&gt;Network training&lt;/td&gt;
      &lt;td&gt;&lt;strong&gt;Dimension-agnostic&lt;/strong&gt;&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;hr /&gt;

&lt;h3 id=&quot;125-summary-the-three-pillars-of-dimension-independence&quot;&gt;12.5 Summary: The Three Pillars of Dimension-Independence&lt;/h3&gt;

&lt;ol&gt;
  &lt;li&gt;&lt;strong&gt;Infinite-Dimensional Hilbert Space&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;Work directly in \(L^2([0,T], \mathbb{R}^{d_a})\) without discretization&lt;/li&gt;
      &lt;li&gt;Avoid the dimensional explosion of finite grids&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Covariance-Weighted Loss&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;Cameron-Martin loss weights errors by \(\mathcal{C}_\mu^{-1/2}\)&lt;/li&gt;
      &lt;li&gt;Automatically accounts for data geometry&lt;/li&gt;
      &lt;li&gt;Prevents high-variance directions from dominating&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Measure-Theoretic Convergence&lt;/strong&gt;
    &lt;ul&gt;
      &lt;li&gt;Backward Kolmogorov equation: deterministic PDE, not stochastic&lt;/li&gt;
      &lt;li&gt;Girsanov + Pinsker: rigorous information-theoretic bounds&lt;/li&gt;
      &lt;li&gt;Constants depend only on \(\operatorname{Tr}(\mathcal{C}_\mu)\), not problem dimension&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ol&gt;
</description>
        <pubDate>Mon, 13 Jul 2026 12:00:00 +0000</pubDate>
        <link>http://scriptedonachip.com/kolmogorov-regress</link>
        <guid isPermaLink="true">http://scriptedonachip.com/kolmogorov-regress</guid>
        
        
        <category>diffusion</category>
        
        <category>control</category>
        
        <category>robotics</category>
        
        <category>optimal-control</category>
        
      </item>
    
      <item>
        <title>Infinite-Dimensional Diffusion Policies via Inverse Kolmogorov PDEs</title>
        <description>
&lt;script type=&quot;text/x-mathjax-config&quot;&gt;
MathJax.Hub.Config({
  TeX: { equationNumbers: { autoNumber: &quot;AMS&quot; } }
});
&lt;/script&gt;

&lt;!--Mathjax Parser --&gt;
&lt;script type=&quot;text/javascript&quot; async=&quot;&quot; src=&quot;https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML&quot;&gt;
&lt;/script&gt;

&lt;script type=&quot;text/x-mathjax-config&quot;&gt;
MathJax.Hub.Config({
  tex2jax: {inlineMath: [[&apos;$&apos;,&apos;$&apos;], [&apos;\\(&apos;,&apos;\\)&apos;]]}
});
&lt;/script&gt;

&lt;blockquote&gt;
  &lt;p&gt;This post accompanies my recent work, &lt;em&gt;Visuomotor Diffusion Policies via Inverse
PDEs&lt;/em&gt; (&lt;a href=&quot;/downloads/Papers/robodiff.pdf&quot;&gt;RoboDiff&lt;/a&gt;). It distills two
companion notes — a first-principles derivation of the &lt;strong&gt;backward Kolmogorov equation
(BKE)&lt;/strong&gt; in infinite dimensions, and a hands-on &lt;strong&gt;PushT&lt;/strong&gt; exposition notebook — into one
narrative. The claim in a sentence: &lt;em&gt;for long-horizon planning, don’t diffuse points —
diffuse functions.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2 id=&quot;why-point-based-diffusion-policies-drift&quot;&gt;Why point-based diffusion policies drift&lt;/h2&gt;

&lt;p&gt;A modern diffusion policy samples an action &lt;strong&gt;chunk&lt;/strong&gt; \(a \in \mathbb{R}^{T_p \times d_a}\) by
running a denoiser backward from Gaussian noise. That works beautifully at short horizons.
But the moment you stack the horizon \(T_p\) high — the regime you actually want for
long-horizon manipulation and planning — the geometry turns against you. You are denoising
in \(\mathbb{R}^{N}\) with \(N = T_p \cdot d_a\), and the standard DDPM convergence bound reads&lt;/p&gt;

\[\|\mu_\theta - \mu_{\text{data}}\|_{\text{TV}} = O\!\left(\sqrt{N}\,\sqrt{\text{loss}}\right).\]

&lt;p&gt;The \(\sqrt{N}\) is the villain. Refine the temporal grid and the bound &lt;em&gt;degrades&lt;/em&gt;. White
noise \(\mathcal{N}(0, I)\) treats every timestep as an independent coordinate, so the sampler
has no notion that a trajectory should be &lt;em&gt;smooth in time&lt;/em&gt;. The result is the familiar
long-horizon drift.&lt;/p&gt;

&lt;p&gt;The fix is not a better network. It is a better &lt;strong&gt;space&lt;/strong&gt;.&lt;/p&gt;

&lt;h2 id=&quot;the-move-diffuse-in-a-hilbert-space&quot;&gt;The move: diffuse in a Hilbert space&lt;/h2&gt;

&lt;p&gt;Instead of an action chunk being a vector, treat it as an element of the function space&lt;/p&gt;

\[\mathcal{H} = L^2\!\left([0,T],\, \mathbb{R}^{d_a}\right),\]

&lt;p&gt;the square-integrable action &lt;em&gt;trajectories&lt;/em&gt;. This is infinite-dimensional from the start —
there is no grid to explode. The whole machinery of diffusion (a forward noising process, a
score, a reverse SDE) lifts to \(\mathcal{H}\), but three objects have to be built with care:
the &lt;strong&gt;noise&lt;/strong&gt;, the &lt;strong&gt;loss&lt;/strong&gt;, and the &lt;strong&gt;PDE&lt;/strong&gt; the denoiser learns.&lt;/p&gt;

&lt;h3 id=&quot;noise-must-be-colored--and-the-color-is-forced-on-us&quot;&gt;Noise must be colored — and the color is forced on us&lt;/h3&gt;

&lt;p&gt;On \(\mathcal{H}\) you cannot use white noise: \(\mathcal{N}(0,I)\) has no meaning when \(I\) is not
trace-class. The admissible noise is a &lt;strong&gt;\(\mathcal{C}\)-Wiener process&lt;/strong&gt;, driven by a
covariance operator \(\mathcal{C}_\mu\) whose Mercer spectrum \(\{(\lambda_k, e_k)\}\) must be
summable:&lt;/p&gt;

\[W_t^{\mathcal{C}} = \sum_{k=1}^\infty \sqrt{\lambda_k}\,\beta_k(t)\,e_k,
\qquad \operatorname{Tr}(\mathcal{C}_\mu) = \sum_{k=1}^\infty \lambda_k &amp;lt; \infty.\]

&lt;p&gt;Each mode \(e_k\) is excited &lt;em&gt;independently&lt;/em&gt; at rate \(\sqrt{\lambda_k}\), and the trace-class
condition \(\sum_k \lambda_k &amp;lt; \infty\) is exactly what keeps sample paths inside \(\mathcal{H}\).
In practice \(\mathcal{C}_\mu\) is a &lt;strong&gt;Matérn-3/2&lt;/strong&gt; temporal kernel; its eigenvalues decay like
\(\lambda_k \sim k^{-3}\), so the noise it injects is temporally &lt;em&gt;smooth&lt;/em&gt; by construction.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/robodiff/kernels.png&quot; alt=&quot;Cameron-Martin covariance kernels and their spectra&quot; /&gt;
&lt;em&gt;Candidate covariance kernels (Matérn-3/2, RBF, Brownian) and their eigenvalue decay
\(\lambda_k\). Smooth kernels concentrate energy in a few low-\(k\) modes — the spectral prior
that a good trajectory should be smooth.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Sampling \(\eta = L\xi\) with \(\xi \sim \mathcal{N}(0,I)\) and \(\mathcal{C}_\mu = LL^\top\) turns
white noise into colored noise. The visual difference is the whole point:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/robodiff/white_vs_colored.png&quot; alt=&quot;White vs. colored noise samples&quot; /&gt;
&lt;em&gt;White noise (left) is jagged and coordinate-wise independent; colored noise
\(\eta \sim \mathcal{N}(0, \mathcal{C}_\mu)\) (right) is temporally coherent. Corrupting a
trajectory with colored noise perturbs it as a &lt;strong&gt;function&lt;/strong&gt;, never off the manifold of
plausible motions.&lt;/em&gt;&lt;/p&gt;

&lt;h3 id=&quot;the-cameron-martin-space-is-the-set-of-legal-perturbations&quot;&gt;The Cameron-Martin space is the set of legal perturbations&lt;/h3&gt;

&lt;p&gt;Why is this the &lt;em&gt;only&lt;/em&gt; correct noise? Because the &lt;strong&gt;Cameron-Martin space&lt;/strong&gt;&lt;/p&gt;

\[\mathcal{H}_{\mathcal{C}} = \operatorname{Range}(\mathcal{C}_\mu^{1/2})
= \Big\{ h : \|h\|_{\mathcal{H}_\mathcal{C}}^2 = \sum_k \tfrac{\hat h_k^2}{\lambda_k} &amp;lt; \infty \Big\}\]

&lt;p&gt;is &lt;em&gt;exactly&lt;/em&gt; the set of directions along which the Gaussian measure \(\mathcal{N}(0,\mathcal{C}_\mu)\)
can be shifted while staying absolutely continuous. Push in a direction outside
\(\mathcal{H}_\mathcal{C}\) — as white noise does at every reverse step — and the measures become
&lt;strong&gt;mutually singular&lt;/strong&gt;: the trajectory instantly leaves the support of the data. Colored noise
is not a cosmetic tweak; it is a measure-theoretic necessity.&lt;/p&gt;

&lt;h2 id=&quot;the-forward-process-decouples-spectrally&quot;&gt;The forward process decouples spectrally&lt;/h2&gt;

&lt;p&gt;The forward dynamics are an Ornstein–Uhlenbeck (OU) SDE on \(\mathcal{H}\),&lt;/p&gt;

\[dX_t = -\tfrac{1}{2}X_t\,dt + dW_t^{\mathcal{C}}, \qquad X_0 \sim \mu_{\text{data}}.\]

&lt;p&gt;Project onto each Mercer mode \(\alpha_k(t) = \langle X_t, e_k\rangle\) and it shatters into
independent scalar OU processes with a closed-form transition law,&lt;/p&gt;

\[d\alpha_k(t) = -\tfrac{1}{2}\alpha_k(t)\,dt + \sqrt{\lambda_k}\,d\beta_k(t),
\qquad
\alpha_k(t)\mid\alpha_k(0) \sim \mathcal{N}\!\big(e^{-t/2}\alpha_k(0),\ \lambda_k(1-e^{-t})\big).\]

&lt;p&gt;The \(\lambda_k\) prefactor on the variance is the signature of colored noise: high-energy
(low-\(k\)) modes are corrupted more, high-\(k\) detail modes barely at all.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/robodiff/spectral_forward.png&quot; alt=&quot;Forward OU process in spectral coordinates&quot; /&gt;
&lt;em&gt;The forward process in spectral coordinates. Each mode \(\alpha_k\) mean-reverts to
\(\mathcal{N}(0,\lambda_k)\) at its own rate — an exact, per-mode Gaussian, no discretization.&lt;/em&gt;&lt;/p&gt;

&lt;h2 id=&quot;the-star-of-the-show-the-backward-kolmogorov-equation&quot;&gt;The star of the show: the backward Kolmogorov equation&lt;/h2&gt;

&lt;p&gt;Here is the conceptual pivot of the paper. Everything a denoiser needs to know lives in the
&lt;strong&gt;value function&lt;/strong&gt;&lt;/p&gt;

\[u(x,s) = \mathbb{E}\big[f(X_t)\mid X_s = x\big],\]

&lt;p&gt;and by Itô’s formula plus the martingale property, \(u\) satisfies a &lt;em&gt;deterministic&lt;/em&gt; PDE — the
backward Kolmogorov equation:&lt;/p&gt;

\[-\frac{\partial u}{\partial s}(x,s)
= \Big\langle -\tfrac{1}{2}x,\ \nabla_x u \Big\rangle_{\mathcal{H}}
+ \tfrac{1}{2}\operatorname{Tr}\!\big[\mathcal{C}_\mu \cdot D^2 u\big],
\qquad u(x,t) = f(x).\]

&lt;p&gt;This reframing is the payoff. The OU &lt;strong&gt;SDE&lt;/strong&gt; describes one noisy path; to get an expectation
out of it you Monte-Carlo. The &lt;strong&gt;BKE&lt;/strong&gt; describes the object the denoiser actually approximates,
and it does so:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;deterministically&lt;/strong&gt; — score matching becomes structured regression against a PDE
solution, not averaging over sampled rollouts;&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;spectrally&lt;/strong&gt; — in the Mercer basis the diffusion term is
\(\tfrac{1}{2}\sum_k \lambda_k\,\partial^2_{e_k} u\), so modes decouple and a two-scale
hierarchy (coarse locomotion vs. fine manipulation) &lt;em&gt;emerges&lt;/em&gt; from the structure rather
than being designed in;&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;checkably&lt;/strong&gt; — the score is recovered as \(\nabla_x \log p_s(x) = \mathcal{C}_\mu^{-1}\nabla_x u\),
with no Lebesgue density ever needed (there isn’t one in infinite dimensions).&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;the-training-change-is-three-lines&quot;&gt;The training change is three lines&lt;/h2&gt;

&lt;p&gt;Everything above collapses into a tiny diff against a standard DDPM policy. The network is
&lt;em&gt;identical&lt;/em&gt;; the magic is in the noise and the loss.&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;&lt;strong&gt;Colored noise:&lt;/strong&gt; &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;η = L @ ξ&lt;/code&gt; instead of &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;ε = ξ&lt;/code&gt;.&lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Cameron-Martin loss:&lt;/strong&gt; penalize the error in the \(\mathcal{H}_\mathcal{C}\)-norm,&lt;/p&gt;

\[\mathcal{L}_{\mathrm{CM}}(\theta)
= \mathbb{E}\!\left[\big\|\mathcal{C}_\mu^{-1/2}\big(\eta_\theta(X_s,s) - \eta\big)\big\|_{\mathcal{H}}^2\right]
= \mathbb{E}\!\left[\sum_{k=1}^\infty \frac{|\hat\eta_{\theta,k} - \hat\eta_k|^2}{\lambda_k}\right].\]
  &lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Colored reverse noise:&lt;/strong&gt; the posterior sample &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;z = L @ ξ&apos;&lt;/code&gt; instead of &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;z = ξ&apos;&lt;/code&gt;.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The \(1/\lambda_k\) weighting is not a hyperparameter — it is the &lt;em&gt;precision&lt;/em&gt; (inverse variance)
of each mode. It heavily penalizes errors in low-energy modes, where the score is sharp and
the denoiser must be exact, and forgives errors in high-energy modes, where the score is gentle.
It is the unique weighting consistent with the Cameron-Martin geometry of \(\mathcal{N}(0,\mathcal{C}_\mu)\).&lt;/p&gt;

&lt;p&gt;A quick sanity check that the colored forward noising is implemented correctly:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/robodiff/forward_process_check.png&quot; alt=&quot;Forward process verification&quot; /&gt;
&lt;em&gt;Empirical vs. analytic forward statistics: the colored forward process matches the
closed-form OU marginals mode-by-mode.&lt;/em&gt;&lt;/p&gt;

&lt;h2 id=&quot;the-headline-result-dimension-independent-convergence&quot;&gt;The headline result: dimension-independent convergence&lt;/h2&gt;

&lt;p&gt;Assembling Mercer → trace-class → Cameron-Martin (measure theory), the OU forward process
(dynamics), and Girsanov + Pinsker (information theory) yields the paper’s main theorem.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;&lt;strong&gt;Theorem 1 (Dimension-Independent Convergence).&lt;/strong&gt; For a policy trained with the
Cameron-Martin loss,&lt;/p&gt;

\[\|\mu_\theta - \mu_{\text{data}}\|_{\text{TV}}
\le C_1\sqrt{\mathcal{L}_{\mathrm{CM}}(\theta)} + C_2\,e^{-T/2},\]

  &lt;p&gt;where \(C_1\) depends only on \(\operatorname{Tr}(\mathcal{C}_\mu)\) and universal constants, and
\(C_2\) only on OU-operator properties. &lt;strong&gt;Neither depends on the action dimension \(d_a\), the
discretization resolution \(N\), nor (except exponentially) the horizon \(T\).&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Contrast the two bounds side by side:&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt; &lt;/th&gt;
      &lt;th&gt;Finite-dim DDPM&lt;/th&gt;
      &lt;th&gt;Infinite-dim (this work)&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;Space&lt;/td&gt;
      &lt;td&gt;grid \(\mathbb{R}^N\), \(N \gg d_a\)&lt;/td&gt;
      &lt;td&gt;\(L^2([0,T],\mathbb{R}^{d_a})\), no grid&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Noise&lt;/td&gt;
      &lt;td&gt;\(\mathcal{N}(0,I)\) (white)&lt;/td&gt;
      &lt;td&gt;\(\mathcal{N}(0,\mathcal{C}_\mu)\) (colored)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Loss&lt;/td&gt;
      &lt;td&gt;\(\mathbb{E}|\eta_\theta-\eta|^2\)&lt;/td&gt;
      &lt;td&gt;\(\mathbb{E}|\mathcal{C}_\mu^{-1/2}(\eta_\theta-\eta)|^2\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;TV bound&lt;/td&gt;
      &lt;td&gt;\(O(\sqrt{N}\sqrt{\text{loss}})\)&lt;/td&gt;
      &lt;td&gt;\(O(\sqrt{\text{loss}})\)&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;The \(\sqrt{N}\) is gone. The constants collapse onto a single fixed scalar — the trace of the
covariance — because the \(\mathcal{C}_\mu^{-1/2}\) weighting absorbs exactly the geometry that
would otherwise leak dimension into the bound.&lt;/p&gt;

&lt;h2 id=&quot;a-physics-aware-diagnostic-for-free&quot;&gt;A physics-aware diagnostic for free&lt;/h2&gt;

&lt;p&gt;Because the true \(u\) solves the BKE &lt;em&gt;exactly&lt;/em&gt;, you can plug the learned \(u_\theta\) back into
the equation and measure the violation — the &lt;strong&gt;Kolmogorov residual&lt;/strong&gt;:&lt;/p&gt;

\[\mathcal{R}_\theta(x,s)
= \frac{\partial u_\theta}{\partial s}
+ \Big\langle -\tfrac{1}{2}x,\ \nabla_x u_\theta \Big\rangle_{\mathcal{H}}
+ \tfrac{1}{2}\operatorname{Tr}\!\big[\mathcal{C}_\mu \cdot D^2 u_\theta\big].\]

&lt;p&gt;This is a &lt;em&gt;pointwise&lt;/em&gt; PDE error, not a statistical one: it needs a single query \((x,s)\) and
automatic differentiation of the network — &lt;strong&gt;no environment rollout, no new data, no task
outcome&lt;/strong&gt;. \(\mathcal{R}_\theta \equiv 0\) iff the model solves the BKE. It is a certificate you
can evaluate at inference time, which is exactly the property you want for high-stakes,
robots-in-production control.&lt;/p&gt;

&lt;h2 id=&quot;does-it-actually-work-results&quot;&gt;Does it actually work? Results&lt;/h2&gt;

&lt;p&gt;Theory is only worth as much as the numbers behind it. The paper validates the framework on
a visuomotor manipulation benchmark and a manufacturing-flow control problem, sweeping the
loss from pure MSE (\(\alpha=0\), the finite-dimensional baseline) through mixed precision to
the full Cameron-Martin objective (\(\alpha=1\)).&lt;/p&gt;

&lt;h3 id=&quot;pusht-manipulation&quot;&gt;PushT manipulation&lt;/h3&gt;

&lt;p&gt;On the &lt;strong&gt;PushT&lt;/strong&gt; benchmark — align a T-shaped block with a 2-DOF pusher from pixels — the
Cameron-Martin policy reaches a &lt;strong&gt;maximum episodic reward of ≈ 0.95 vs. ≈ 0.78 for the MSE
baseline, a 17% improvement&lt;/strong&gt;, and cuts inter-step trajectory drift by &lt;strong&gt;67.6%&lt;/strong&gt;. The gain is
sharpest in the final approach phase (steps 300–400): the MSE policy develops high-frequency
oscillations that destabilize contact, while the CM policy stays smooth.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/robodiff/paper_pusht_rewards.png&quot; alt=&quot;PushT inference rewards across loss formulations&quot; /&gt;
&lt;em&gt;Figure 2 from the paper. Max and mean episodic reward as the loss interpolates from MSE
(\(\mathcal{L}_{\text{MSE}}\)) through mixed precision (\(\mathcal{L}_{\text{MP}}\)) to
Cameron-Martin (\(\mathcal{L}_{\text{CM}}\)). Full measure-theoretic weighting tops out at
reward 1.0.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The &lt;strong&gt;Kolmogorov residual&lt;/strong&gt; tracks this cleanly, monotonically decreasing as the precision
weighting \(\alpha\) turns on:&lt;/p&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;Loss (\(\alpha\))&lt;/th&gt;
      &lt;th&gt;Mean residual \(\mathcal{R}\)&lt;/th&gt;
      &lt;th&gt;Reduction vs. MSE&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;MSE (\(0\))&lt;/td&gt;
      &lt;td&gt;0.34&lt;/td&gt;
      &lt;td&gt;—&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Mixed (\(0.15\))&lt;/td&gt;
      &lt;td&gt;0.28&lt;/td&gt;
      &lt;td&gt;17.6%&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Mixed (\(0.25,\,0.50\))&lt;/td&gt;
      &lt;td&gt;0.22, 0.18&lt;/td&gt;
      &lt;td&gt;—&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;Cameron-Martin (\(1.0\))&lt;/td&gt;
      &lt;td&gt;&lt;strong&gt;0.11&lt;/strong&gt;&lt;/td&gt;
      &lt;td&gt;&lt;strong&gt;67.6%&lt;/strong&gt;&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;p&gt;And the residual doubles as an &lt;strong&gt;oracle-free failure detector&lt;/strong&gt;: rollouts with mean residual
\(&amp;gt; 0.15\) correlate with task failure (max reward \(&amp;lt; 0.5\)), while those below \(0.10\) succeed
\(&amp;gt; 90\%\) of the time — a real-time trustworthiness signal that needs no ground-truth reward.&lt;/p&gt;

&lt;h3 id=&quot;manufacturing-flow-conwip&quot;&gt;Manufacturing flow (CONWIP)&lt;/h3&gt;

&lt;p&gt;The same function-space machinery transfers to a stochastic &lt;strong&gt;6-station manufacturing line&lt;/strong&gt;
under CONWIP flow control, where queue dynamics are genuinely function-valued. Against an
&lt;strong&gt;LSTM + CONWIP&lt;/strong&gt; baseline, the infinite-dimensional policy cuts normalized work-in-process
&lt;strong&gt;RMSE by 28.4%&lt;/strong&gt; with high starvation-event recall (≈ 0.98).&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/robodiff/paper_rmse_lstm.png&quot; alt=&quot;InfDiff vs. LSTM baseline RMSE&quot; /&gt;
&lt;em&gt;Figure 8 from the paper. Training/validation curves for the infinite-dimensional scheme
(left) vs. the LSTM baseline (right); the InfDiff floor sits at \(\varepsilon = 0.105\).&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Two results stand out. First, the Kolmogorov residual &lt;strong&gt;localizes the bottleneck station&lt;/strong&gt;
with a &lt;strong&gt;13× signal-to-noise ratio&lt;/strong&gt; (Precision@1 = 1.0) and sub-second latency — an anomaly
detector the LSTM can only infer indirectly. Second, seeding a &lt;strong&gt;Hamilton-Jacobi reachability&lt;/strong&gt;
computation (&lt;a href=&quot;https://github.com/robotsorcerer/levelsetpy&quot;&gt;LevelSetPy&lt;/a&gt;) with the InfDiff
forecast yields a certified dispatch filter that &lt;strong&gt;prevents 351 deadlock events across 100
runs — a 96% reduction&lt;/strong&gt; — keeping every station above starvation and below the CONWIP cap.&lt;/p&gt;

&lt;h2 id=&quot;takeaways&quot;&gt;Takeaways&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;strong&gt;Diffuse functions, not points.&lt;/strong&gt; Lifting action chunks to \(L^2([0,T])\) removes the grid
whose refinement was hurting long-horizon policies.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;The color of the noise is forced.&lt;/strong&gt; The Cameron-Martin space is the exact set of legal
perturbations; colored \(\mathcal{N}(0,\mathcal{C}_\mu)\) noise is a measure-theoretic
necessity, not a trick.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Learn the PDE, not the paths.&lt;/strong&gt; The backward Kolmogorov equation is the deterministic,
spectral, checkable object the denoiser truly approximates.&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Get a theorem out of it.&lt;/strong&gt; Cameron-Martin weighting buys dimension-independent
convergence, plus a physics-aware residual you can certify at inference.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;All of this costs three lines of code over a standard diffusion policy.&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;&lt;strong&gt;Read more.&lt;/strong&gt; The full paper is
&lt;a href=&quot;/downloads/Papers/robodiff.pdf&quot;&gt;here&lt;/a&gt;; the derivations condense two
companion notes (a first-principles BKE derivation and a PushT exposition notebook). Questions
and pushback welcome in the comments.&lt;/p&gt;
</description>
        <pubDate>Sun, 24 May 2026 09:00:00 +0000</pubDate>
        <link>http://scriptedonachip.com/infinite-dimensional-diffusion-policies</link>
        <guid isPermaLink="true">http://scriptedonachip.com/infinite-dimensional-diffusion-policies</guid>
        
        
        <category>diffusion</category>
        
        <category>control</category>
        
        <category>robotics</category>
        
        <category>optimal-control</category>
        
      </item>
    
      <item>
        <title>Stochastic Policies, Diffusion, Flow matching</title>
        <description>
&lt;script type=&quot;text/x-mathjax-config&quot;&gt;
MathJax.Hub.Config({
  TeX: { equationNumbers: { autoNumber: &quot;AMS&quot; } }
});
&lt;/script&gt;

&lt;!--Mathjax Parser --&gt;
&lt;script type=&quot;text/javascript&quot; async=&quot;&quot; src=&quot;https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML&quot;&gt;
&lt;/script&gt;

&lt;script type=&quot;text/x-mathjax-config&quot;&gt;
MathJax.Hub.Config({
  tex2jax: {inlineMath: [[&apos;$&apos;,&apos;$&apos;], [&apos;\\(&apos;,&apos;\\)&apos;]]}
});
&lt;/script&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://proceedings.neurips.cc/paper_files/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf&quot;&gt;Denoising Diffusion Probabilistic Models, Jonathan Ho, Ajay Jain, Pieter Abbeel&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/2303.04137&quot;&gt;Diffusion Policy: Visuomotor Policy Learning via Action Diffusion, Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake 2,3, Shuran Song&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/2407.12381v2&quot;&gt;Flow Matching Imitation Learning for Multi-Support Manipulation, Quentin Rouxel, Andrea Ferrari, Serena Ivaldi, and Jean-Baptiste Mouret&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;#https://arxiv.org/pdf/2209.03003&quot;&gt;Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow, Xingchao Liu and Chengyue Gong&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/2210.02747&quot;&gt;Flow Matching for Generative Modeling, Yaron Lipman,Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel1 Matt Le&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/2209.15571&quot;&gt;Building Normalizing Flows with Stochastic Interpolants, Michael S. Albergo and Eric Vanden-Eijnden&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/2303.08797&quot;&gt;Stochastic Interpolants: A Unifying Framework for Flows and Diffusions, Michael S. Albergo, Nicholas M. Boffi, and Eric Vanden-Eijnden&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;
</description>
        <pubDate>Fri, 01 Aug 2025 00:00:00 +0000</pubDate>
        <link>http://scriptedonachip.com/stochastic-diffusion</link>
        <guid isPermaLink="true">http://scriptedonachip.com/stochastic-diffusion</guid>
        
        
        <category>control</category>
        
        <category>optimal-control</category>
        
      </item>
    
      <item>
        <title>Optimality vs. Stability of Feedback Control Systems.</title>
        <description>
&lt;script type=&quot;text/x-mathjax-config&quot;&gt;
MathJax.Hub.Config({
  TeX: { equationNumbers: { autoNumber: &quot;AMS&quot; } }
});
&lt;/script&gt;

&lt;!--Mathjax Parser --&gt;
&lt;script type=&quot;text/javascript&quot; async=&quot;&quot; src=&quot;https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML&quot;&gt;
&lt;/script&gt;

&lt;script type=&quot;text/x-mathjax-config&quot;&gt;
MathJax.Hub.Config({
  tex2jax: {inlineMath: [[&apos;$&apos;,&apos;$&apos;], [&apos;\\(&apos;,&apos;\\)&apos;]]}
});
&lt;/script&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;a href=&quot;#table-o-conts&quot;&gt;Table of Contents&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#intro&quot;&gt;Introduction&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#case&quot;&gt;Stability vs optimality&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#issues&quot;&gt;Optimality&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a name=&quot;intro&quot;&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h3 id=&quot;intro&quot;&gt;Intro&lt;/h3&gt;

&lt;p&gt;The question of the connection between optimality and stability is a curious thing.  On the one hand, we are led to believe that if we can find an optimal control law, that can execute a plan, with the least amount of energy possible, then we are satisfied to know we have fulfilled the specifications posed in  our objective function. But consider: an optimal controller is not necessarily a stable controller for a system. Why so? I lay out my case in the next section.&lt;/p&gt;

&lt;p&gt;&lt;a name=&quot;case&quot;&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h3 id=&quot;stability-vs-optimality&quot;&gt;Stability vs optimality&lt;/h3&gt;
&lt;p&gt;Systems under the influence of optimal control laws enjoy a nice set of properties, provided that the associated cost functional enforces a constraint that is desirable on the state and control. LQ optimal control systems have nice gain and phase margins coupled with reduced sensitivity and I understand that there are similar properties that have been shown for nonlinear systems. Optimal control has the attractive property that the control effort is not wasted in mitigating the effects of nonlinearities as it chooses among a set of policies (or stabilizing control sequences) that yield a desirable effect on the system. The intractability of the HJB equation however makes optimal control as a synthesis tool for nonlinear problems a painful one.&lt;/p&gt;

&lt;p&gt;Enter Lyapunov stability. Lyapunov defines classical stability as the system’s behavior near an equilibrium point such that there exists a real number \(\delta(\epsilon, t_0)&amp;gt;0\) for every real \(\epsilon &amp;gt; 0\) for which the state of the system is bound – essentially a local stability concept, a scalar bound that expresses how far away a system could ever get from the equilibrium (based on how far away it started).  As Engineers, we do not want to limit ourselves to this local stability context. We want every motion starting sufficiently close to the equilibrium state to converge to the equilibrium as time approaches &lt;em&gt;ad infinitum&lt;/em&gt;. Asymptotic stability captures this need. But again, asymptotic stability is as well a local concept since we do not know &lt;em&gt;a priori&lt;/em&gt; how much magnitude we want for the bound.  Enter equiasymptotic stability in the large. For an \(r&amp;gt;0\) that is fixed and arbitrarily large, we find that as \(t \rightarrow \infty\), all motions converge to the equilibrium uniformly in the initial state from which they start for \(|x_0| \le r \).&lt;/p&gt;

&lt;p&gt;Note that all these definitions merely impose a constraint on the behavior of the states as they evolve over the trajectories of the system. That begs the question, can a control law be stable, yet not optimal (or vice versa)? I think so. Why?&lt;/p&gt;

&lt;p&gt;&lt;a name=&quot;issues&quot;&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h3 id=&quot;optimality&quot;&gt;Optimality&lt;/h3&gt;

&lt;blockquote&gt;
  &lt;p&gt;This section has an update based on what I found from Freeman and Kokotovic’s 1996 Paper in the Int’l J. Optimal &amp;amp; Control. Please skip to the &lt;a href=&quot;#updatedOptStab&quot;&gt;updated part&lt;/a&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Optimality, as Bellman would have us think, deals with reaching the goal state with as minimal an energy as possible. I would think that the principle of optimality and Lyapunov stability have a fundamental disconnect. It seems to me that we may find an optimal control law that is not stable (i.e it’s V(x) gradient function does not strictly decrease along the trajectories of the solution to the dynamic system’s differential equation).&lt;/p&gt;

&lt;p&gt;&lt;del&gt;To buttress this fact, consider that the concepts of stability and optimality appeared in the consciousness of control theorists at two distinct and disconnected eras (or so to say) in history. On the one hand, Lyapunov’s thesis got published in the Soviet union in the 1890’s but his work was not available in English until 1947. Even so, western researchers did not adequately grasp its usefulness until Kalman’s 1960 seminal paper on the second method of Lyapunov. Meanwhile, Bellman’s last formal work on DP and applied DP did not become published until 1962. What is more intriguing is that not anywhere in Bellman’s stability tests (as far as I can tell from what I have read from his books) did he use Lyapunov analyses’ rigor to establish the stability of his principle of optimality methods. Kalman, remarked in his paper in 1960 that  few researchers were aware of Lyapunov methods. We can make a fairly accurate “guesstimation” that had Bellman been aware of Lyapunov’s analyses earlier, it might have creeped into his optimality analyses.&lt;/del&gt;&lt;/p&gt;

&lt;p&gt;&lt;del&gt;I had an exchange with someone about this a while ago, and I am quoting the caveats they expressed in their agreement with my observation below.&lt;/del&gt;&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;&lt;del&gt;1) If optimality is concerned only with the cost from initial condition to final condition, a control law that makes the system unstable might be desirable as unstable systems tend to be very fast.&lt;/del&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;blockquote&gt;
  &lt;p&gt;&lt;del&gt;2) The problem is what happens when you reach the final condition?  An unstable system will not stop there, but will overshoot the goal and go off to infinity.  So you must have the ability to switch to a stabilizing controller when you reach the goal.&lt;/del&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;blockquote&gt;
  &lt;p&gt;&lt;del&gt;An example is in fighter aircraft.  I understand that they become unstable during certain maneuvers such as tight turns so they can move very fast, but then “catch” themselves and stabilize before going too far from the equilibrium.&lt;/del&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a name=&quot;updatedOptStab&quot;&gt;&lt;/a&gt;
&lt;strong&gt;UPDATE [Aug 28, 2018]&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Most of the discussions below are drawn from Freeman and Kokotovic’s &lt;sup id=&quot;fnref:Freeman_Kokotovic&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:Freeman_Kokotovic&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;1&lt;/a&gt;&lt;/sup&gt; 1996 work on &lt;em&gt;point-wise min-norm control laws for robust control lyapunov functions&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;They provide an optimality-based method for choosing a &lt;em&gt;stabilizing&lt;/em&gt; control law once an rclf is known without resorting to cancellation or domination of nonlinear terms, which do not necessarily possess the desirable properties of optimality and may lead to poor robustness and wasted control effort.&lt;/p&gt;

&lt;p&gt;The value function for a meaningful optimal stabilization problem is a Lyapunov function for the closed-loop system.&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;Every meaningful value function is a Lyapunov function (Freeman and Kokotovic, 1996). Every Lyapunov function for every stable closed-loop system is also a value function for a meaningful optimal stabilization problem.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Every Lyapunov function is a meaningful value function&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Both bullets above are important since the first point helps with the analysis of the stability of an optimal feedback control system, while the second link will have implications for their synthesis.&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;Every robust control lyapunov function (rclf) is a meaningful upper value function&lt;/p&gt;

    &lt;ul&gt;
      &lt;li&gt;Every rclf solves the Hamilton Jacobi Isaacs equation associated with a meaningful game. For a known rclf, a feedback law that is optimal w.r.t a meaningful cost functional can be constructed. Matter-of-factly, this can be accomplished without solving the HJI equation for the upper value function or without constructing a cost functional as the optimal feedback can be directly calculated from the rclf without recourse to the HJI equation. Such control laws are called &lt;em&gt;pointwise min-norm&lt;/em&gt; control laws and each one inherits the desirable properties of optimality because &lt;em&gt;every pointwise min-norm control law is optimal for a meaningful game&lt;/em&gt;.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Essentially, this task is an &lt;em&gt;inverse optimal stabilization problem&lt;/em&gt; where for LTI systems, the solution involves choosing a candidate value function and then constructing a meaningful cost functional in order to make the HJB equation valid. For open-loop stable nonlinear systems, one can find a solution by choosing the candidate value function as a Lyapunov function for the open-loop system. For openloop &lt;em&gt;unstable&lt;/em&gt; systems, one can choose a candidate value function as a clf for the system. In Freeman and Kokotovic’s &lt;sup id=&quot;fnref:Freeman_Kokotovic:1&quot; role=&quot;doc-noteref&quot;&gt;&lt;a href=&quot;#fn:Freeman_Kokotovic&quot; class=&quot;footnote&quot; rel=&quot;footnote&quot;&gt;1&lt;/a&gt;&lt;/sup&gt;, actually the authors solve the inverse optimal &lt;em&gt;robust&lt;/em&gt; stabilization problem for systems with disturbances and showed that evert rclf is an upper value function for a meaningful differential game.&lt;/p&gt;

&lt;div class=&quot;footnotes&quot; role=&quot;doc-endnotes&quot;&gt;
  &lt;ol&gt;
    &lt;li id=&quot;fn:Freeman_Kokotovic&quot; role=&quot;doc-endnote&quot;&gt;
      &lt;p&gt;Freeman, R. A., &amp;amp; Kokotovic, P. V. (1996). Inverse Optimality in Robust Stabilization. SIAM Journal on Control and Optimization, 34(4), 1365–1391. https://doi.org/10.1137/S0363012993258732 &lt;a href=&quot;#fnref:Freeman_Kokotovic&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;/a&gt; &lt;a href=&quot;#fnref:Freeman_Kokotovic:1&quot; class=&quot;reversefootnote&quot; role=&quot;doc-backlink&quot;&gt;&amp;#8617;&lt;sup&gt;2&lt;/sup&gt;&lt;/a&gt;&lt;/p&gt;
    &lt;/li&gt;
  &lt;/ol&gt;
&lt;/div&gt;
</description>
        <pubDate>Wed, 22 Aug 2018 13:10:00 +0000</pubDate>
        <link>http://scriptedonachip.com/opti-stable</link>
        <guid isPermaLink="true">http://scriptedonachip.com/opti-stable</guid>
        
        
        <category>control</category>
        
        <category>stability</category>
        
        <category>optimality</category>
        
      </item>
    
      <item>
        <title>Control Commons</title>
        <description>
&lt;script type=&quot;text/x-mathjax-config&quot;&gt;
MathJax.Hub.Config({
  TeX: { equationNumbers: { autoNumber: &quot;AMS&quot; } }
});
&lt;/script&gt;

&lt;!--Mathjax Parser --&gt;
&lt;script type=&quot;text/javascript&quot; async=&quot;&quot; src=&quot;https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML&quot;&gt;
&lt;/script&gt;

&lt;script type=&quot;text/x-mathjax-config&quot;&gt;
MathJax.Hub.Config({
  tex2jax: {inlineMath: [[&apos;$&apos;,&apos;$&apos;], [&apos;\\(&apos;,&apos;\\)&apos;]]}
});
&lt;/script&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;a href=&quot;#table-o-conts&quot;&gt;Table of Contents&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#intro&quot;&gt;Introduction&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#defs&quot;&gt;Definitions, Theorems, Lemmas etc&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#nlnr&quot;&gt;Nonlinear Control Theory&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;#stab&quot;&gt;Stability&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a name=&quot;intro&quot;&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h3 id=&quot;intro&quot;&gt;Intro&lt;/h3&gt;

&lt;p&gt;Here are a few control theorems, concepts and diagrams that I think every control student should know. I keep updating this post, so please check back from time to time.&lt;/p&gt;

&lt;p&gt;&lt;a name=&quot;defs&quot;&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h3 id=&quot;definitions-theorems-lemmas-and-such&quot;&gt;Definitions, Theorems, Lemmas and such.&lt;/h3&gt;

&lt;p&gt;&lt;a name=&quot;nlnr&quot;&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h4 id=&quot;nonlinear-control-theory&quot;&gt;Nonlinear Control Theory&lt;/h4&gt;

&lt;ul&gt;
  &lt;li&gt;A differential equation of the form&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;\begin{align}
  dx/dt = f(x, u(t), t), \quad -\infty &amp;lt; t &amp;lt; +\infty
  \label{eq:diff_eq}
\end{align}&lt;/p&gt;

&lt;p&gt;is said to be free (or unforced) if \(u(t) \equiv 0\) for all \(t\). That is \eqref{eq:diff_eq} becomes&lt;/p&gt;

&lt;p&gt;\begin{align}
  dx/dt = f(x, t), \quad -\infty &amp;lt; t &amp;lt; +\infty
  \label{eq:unforced}
\end{align}&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;If the differential equation in \eqref{eq:diff_eq} does not have an explicit dependence on time, but has an implicit dependence on time, through \(u(t)\), then the system is said to be stationary. In other words, a dynamic system is &lt;strong&gt;stationary&lt;/strong&gt; if&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;\begin{align}
  f(x, u(t), t) \equiv f(x, u(t))
  \label{eq:stationary}
\end{align}&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;A stationary system \eqref{eq:stationary} that is free is said to be &lt;em&gt;invariant under time translation&lt;/em&gt;, i.e.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;\begin{align}
  \Phi(t; x_0, t_0) = \Phi(t + \tau; x_0, t_0 + \tau)
  \label{eq:free_stat}
\end{align}
    - \(\Phi(t; x_0, t_0)\) is the analytical solution to \eqref{eq:diff_eq}; it is generally interpreted as the solution of \eqref{eq:diff_eq}, with fixed \(u(t)\), going through state \(x_0\) at time \(t_0\) and observed at time \(t\) later on. This is a clearer way of representing the d.e.’s solution as against \(x(t)\), which is popularly used in most text nowadays.&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;\(\Phi(\cdot)\) is generally referred to the transition function, since it relates the transformation from \(x(t_0)\) to \(x(t)\).&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;For a physical system, \(\Phi\) has to be &lt;em&gt;continuous in all of its arguments.&lt;/em&gt;
.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;If the rate of change \(dE(x)/dx\) of an isolated physical system is negative for every possible state x, except for a single equilibrium state \(x_e\), then the energy will continually decrease until it finally assumes its minimum value \(E(x)\).&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;The &lt;strong&gt;first method of Lyapunov&lt;/strong&gt; deals with questions of stability using an explicit representation of the solutions of a differential equation
    &lt;ul&gt;
      &lt;li&gt;Note that the &lt;strong&gt;second method&lt;/strong&gt; is more of a historical misnomer, perhaps more accurately described as a philosophical point of view rather than a systematic method. Successful application requires the user’s ingenuity.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;In contrast to popular belief that the energy of a system and a Lyapunov function are the same, they are not the same. Why? Because &lt;strong&gt;the Lyapunov function, \(V(x)\), is not unique&lt;/strong&gt;. To quote Kalman, “a system whose energy \(E\) decreases &lt;em&gt;on the average&lt;/em&gt;, but not necessarily at each instant, is stable but \(E\) is not necessarily a Lyapunov function.”&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Lyapunov analysis and optimization&lt;/strong&gt;: Suppose a performance index is defined to be the error criterion between a measured and an estimated signal; suppose further that this criterion is integrated w.r.t time, then the performance index is actually a Lyapunov function – provided that the error is not identically zero along any trajectory of the system.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;strong&gt;Existence, uniqueness, and continuity theorem&lt;/strong&gt;:&lt;/p&gt;

    &lt;p&gt;Let \(f(x, t)\) be continuous in \(x,t\), and satisfy a Lipschitz condition in some region about any state \(x_0\) passing through time \(t_0\):&lt;/p&gt;

    &lt;p&gt;\begin{align}
  R(x_0, t_0) &amp;amp;=
    ||x - x_0|| \le b(x_0) \nonumber
\end{align}&lt;/p&gt;

    &lt;p&gt;\begin{align}
  R(x_0, t_0) &amp;amp;= ||t - t_0|| \le c(t_0)  \quad (b, c) &amp;gt; 0
\end{align}&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;with the Lipschitz condition satisfied for \((x,t), (y,t)\) \(\in\) \(R(x_0, t_0)\), then it follows that
\begin{align}
  ||f(x,t) - f(y,t)|| \le k \, ||x-y|| \nonumber
\end{align}&lt;/p&gt;

&lt;p&gt;where \(k&amp;gt;0\) depends on \(b, c\). THUS,&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;there exists a unique solution \(\Phi(t; x_0, t_0)\) of \(dx/dt\), that starts as \(x_0, t_0\) for all \(|t - t_0| \le a(t_0)\),&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;\(a(t_0) \ge \text{ Min (}{c(t_0), b(x(t_0))/M(x_0, t_0)}\), where \(M(x_0, t_0)\) is the maximum assumed by the continuous function \(f(x,t)\) in the closed, bounded set \(R(x_0, t_0)\)&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;in some small neighborhood of \(x_0, t_0\), the solution is continuous in its arguments&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Observe that the Lipschitz condition only implies continuity of \(f\) in \(x\) but not necessarily in \(t\); as it is implied by the bounded derivatives in \(x\). Note that the local lipschitz condition required by the theorem only implies desired properties of a solution near \(x_0, t_0\).&lt;/p&gt;

&lt;p&gt;The &lt;em&gt;finite escape time&lt;/em&gt; (that is the solution leaves any compact set within a finite time) quandary does not allow us to make conclusions surrounding arbitrarily large values of \(t\). The phrase “&lt;strong&gt;finite escape time&lt;/strong&gt;” describes the concept that a trajectory escapes to infinity at a finite time. &lt;strong&gt;In order that a differential equation accurately represent a physical system, the possibility of finite escape time has to be mitigated by an explicit assumption to the contrary.&lt;/strong&gt; If the Lipschitz condition holds for \(f\) everywhere, then there can be no finite escape time. The proof is easy by integrating both sides of \eqref{eq:diff_eq} and using&lt;/p&gt;

&lt;p&gt;\begin{align}
  \Phi(t; x_0, t_0) \le ||x_0|| + || \int_{t_0}^{t}f(\Phi(\tau; x_0, t_0), \tau)d\tau ||
\end{align}&lt;/p&gt;

&lt;p&gt;\begin{align}
||x_0|| + k \int_{t_0}^{t}f(\Phi(\tau; x_0, t_0), \tau)d\tau
\end{align}&lt;/p&gt;

&lt;p&gt;where \(f(\cdot)\) obeys the lipschitz condition,&lt;/p&gt;

&lt;p&gt;\begin{align}
  ||f(x,t) - f(y,t)|| \le k \, ||x-y||. \nonumber
\end{align}&lt;/p&gt;

&lt;p&gt;By the Gronwall-Bellman lemma,&lt;/p&gt;

&lt;p&gt;\begin{align}
  ||\Phi(t; x_0, t_0) ||  \le [\exp \, k (t - t_0)] ||x_0 || \nonumber
\end{align}&lt;/p&gt;

&lt;p&gt;which is less than \(\infty \) for any finite \((t - t_0)\).&lt;/p&gt;

&lt;p&gt;&lt;a name=&quot;stab&quot;&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h3 id=&quot;stability&quot;&gt;Stability&lt;/h3&gt;

&lt;p&gt;My definitions follow from R.E Kalman’s 1960 seminal paper since they are clearer to understand compared to the myriad of definitions that exist in many texts today. &lt;strong&gt;Stability concerns the deviation about some fixed motion&lt;/strong&gt;. So, we will be considering the deviations from the equilibrium state \(x_e\) of a free dynamic system.&lt;/p&gt;

&lt;p&gt;Simply put, here is how Kalman defines stability, if \eqref{eq:diff_eq} is slightly perturbed from its equilibrium state at the origin, all subsequent motions remain in a correspondingly small neighborhood of the origin. Harmonic oscillators are a good example of this kind of stability. &lt;strong&gt;Lyapunov&lt;/strong&gt; himself defines stability like so:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;table&gt;
      &lt;tbody&gt;
        &lt;tr&gt;
          &lt;td&gt;An equilibrium state \(x_e\) of a free dynamic system ios &lt;em&gt;stable&lt;/em&gt; id for every real number \(\epsilon&amp;gt;0\), there exists a real number \(\delta(\epsilon, t_0)&amp;gt;0\) such that \(&lt;/td&gt;
          &lt;td&gt; &lt;/td&gt;
          &lt;td&gt;x_0 - x_e&lt;/td&gt;
          &lt;td&gt; &lt;/td&gt;
          &lt;td&gt;\le \delta \) implies&lt;/td&gt;
        &lt;/tr&gt;
      &lt;/tbody&gt;
    &lt;/table&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;\begin{align}
  ||\Phi(t; x_0, t_0) - x_e|| \le \epsilon \quad \forall \quad t \ge t_0 \nonumber
\end{align}&lt;/p&gt;

&lt;p&gt;This is best imagined from the figure below:&lt;/p&gt;

&lt;div class=&quot;fig figcenter fighighlight&quot;&gt;
  &lt;img src=&quot;/assets/control/stability.png&quot; width=&quot;100%&quot; height=&quot;450&quot; align=&quot;middle&quot; /&gt;  
  &lt;div class=&quot;figcaption&quot; align=&quot;middle&quot;&gt;Fig. 1. The basic concept of stability. Courtesy of R.E. Kalman
  &lt;/div&gt;
&lt;/div&gt;

&lt;p&gt;Put differently, the system trajectory can be kept arbitrarily close to the origin/equilibrioum if we start the trajectory sufficiently close to it. If there is stability at some initial time, \(t_0\), there is stability for any other initial time \(t_1\), provided that all motions are continuous in the initial state.&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Asymptotic stability: The requirement that we start sufficiently close to the origin and stay in the neighborhood of the origin is a rather limiting one in most practical engineering applications. We would want to require that our motion should return to equilibrium after any small perturbation. Thus, the classical definition of Lyapunov stability is
    &lt;ul&gt;
      &lt;li&gt;an equilibrium state \(x_e\) of a free dynamic system is &lt;em&gt;asymptotically stable&lt;/em&gt; if
        &lt;ul&gt;
          &lt;li&gt;it is stable and&lt;/li&gt;
          &lt;li&gt;every motion starting sufficiently near \(x_e\) converges to \(x_e\) as \(t \rightarrow \infty\).&lt;/li&gt;
        &lt;/ul&gt;
      &lt;/li&gt;
      &lt;li&gt;put differently, there is some real constant \(r(t_0)&amp;gt;0\) and to every real number
\(\mu &amp;gt; 0\) there corresponds a real number \(T(\mu, x_0, t_0)\) such that \(||x_0 - x_e|| \le r(t_0)\) implies&lt;/li&gt;
    &lt;/ul&gt;

    &lt;p&gt;\begin{align}
  ||\Phi(T; x_0, t_0)|| \le \mu \quad \forall \quad t \ge t_0 + T \nonumber
\end{align}&lt;/p&gt;

    &lt;div class=&quot;fig figcenter fighighlight&quot;&gt;
  &lt;img src=&quot;/assets/control/asymptotic_stability.png&quot; width=&quot;80%&quot; height=&quot;350&quot; align=&quot;middle&quot; /&gt;  
  &lt;div class=&quot;figcaption&quot; align=&quot;left&quot;&gt;Fig. 1. Definition of asymptotic stability. Courtesy of R.E. Kalman
  &lt;/div&gt;
&lt;/div&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Asymptotic stability is also a local concept since we do not know aforetime how small \(r(t_0)\) should be. For motions starting at the same distance from \(x_e\), none will remain at a larger distance than \(\mu\) from \(x\) at arbitrarily large values of time. Or to use Massera’s definition:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;An equilibrium state \(x_e\) of a free dynamic system is &lt;em&gt;equiasymptotically stable&lt;/em&gt; if
    &lt;ul&gt;
      &lt;li&gt;it is stable&lt;/li&gt;
      &lt;li&gt;
        &lt;p&gt;every motion starting sufficiently near \(x_e\) converges to \(x\), as \(t \rightarrow \infty\) uniformly in \(x_0\)&lt;/p&gt;
      &lt;/li&gt;
      &lt;li&gt;
        &lt;p&gt;Interrelations between stability concepts: This I gleaned from Kalman’s 1960 paper on the second method of Lyapunov.&lt;/p&gt;

        &lt;div class=&quot;fig figcenter fighighlight&quot;&gt;
    &lt;img src=&quot;/assets/control/control_concepts.png&quot; width=&quot;100%&quot; height=&quot;450&quot; align=&quot;middle&quot; /&gt;  
    &lt;div class=&quot;figcaption&quot; align=&quot;middle&quot;&gt;Fig. 1. Interrelations between stability concepts. Courtesy of R.E. Kalman
    &lt;/div&gt;
  &lt;/div&gt;
      &lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;For &lt;em&gt;linear systems&lt;/em&gt;, stability is independent of the distance of the initial state from \(x_e\). Nicely defined as such:
    &lt;ul&gt;
      &lt;li&gt;an equilibrium state \(x_e\) of a free dynamic system is &lt;em&gt;asymptotically (equiasymptotically) stable in the large&lt;/em&gt; if
  (i) it is stable&lt;/li&gt;
    &lt;/ul&gt;

    &lt;p&gt;(ii) every motion converges to \(x_e\) as \(t \rightarrow \infty \), i.e., every motion converges to \(x_e\), uniformluy in \(x_0\) for \(x_0 \le r\), where \(r\) is fixed but arbitrarily large&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;To be Continued&lt;/strong&gt;&lt;/p&gt;
</description>
        <pubDate>Sat, 04 Aug 2018 13:10:00 +0000</pubDate>
        <link>http://scriptedonachip.com/control-commons</link>
        <guid isPermaLink="true">http://scriptedonachip.com/control-commons</guid>
        
        
        <category>control</category>
        
        <category>stability</category>
        
        <category>nonlinear-control</category>
        
      </item>
    
      <item>
        <title>What Good Research is Not</title>
        <description>&lt;p&gt;This post includes curated links to advice on doing good research from people I respect.&lt;/p&gt;

&lt;p&gt;I hope you find time to enjoy reading them.&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://www.cc.gatech.edu/~parikh/citizenofcvpr/static/slides/freeman_how_to_write_papers.pdf&quot;&gt;How to write a good research paper ~ Bill Freiman&lt;/a&gt;.&lt;/p&gt;

    &lt;blockquote&gt;
      &lt;p&gt;Many readers will skim over formulas on their first reading of your exposition. Therefore, your sentences should flow smoothly when all but the simplest formulas are replaced by “blah” or some other grunting noise.&lt;/p&gt;
    &lt;/blockquote&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;http://people.csail.mit.edu/billf/publications/How_To_Do_Research.pdf&quot;&gt;How to do good research ~ Bill Freiman&lt;/a&gt;.&lt;/p&gt;

    &lt;blockquote&gt;
      &lt;p&gt;Sometimes it’s useful to think that everyone else is an idiot. This lets you do things that no one else is doing. It’s best not to be too vocal about that. You can say something like “Oh, I just thought I’d try out this direction”.&lt;/p&gt;
    &lt;/blockquote&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;http://people.csail.mit.edu/billf/talks/10minFreeman2013.pdf&quot;&gt;Elements of a Successful Graduate Career&lt;/a&gt;.&lt;/p&gt;

    &lt;blockquote&gt;
      &lt;p&gt;I think the most important thing in research is a story – not a theorem or an algorithm – but the story that makes the theorem or algorithm interesting and exciting.   It’s important to have an “ear” for a good story… when do the stories make sense, when are they bogus? ~ Tomas Lozano-Perez.&lt;/p&gt;
    &lt;/blockquote&gt;

    &lt;blockquote&gt;
      &lt;p&gt;The best students are possessed by a problem. They’re independent. They teach their advisors. They don’t do what they’re told…they do something more interesting. ~ Leslie Kaelbling.&lt;/p&gt;
    &lt;/blockquote&gt;

    &lt;blockquote&gt;
      &lt;p&gt;Don’t tell your advisor you’re doing what they advised against until you’ve solved the problem. ~ Manolis Kellis.&lt;/p&gt;
    &lt;/blockquote&gt;

    &lt;blockquote&gt;
      &lt;p&gt;Which brings us to the moral of the story: More important than your thesis topic is who your advisor is. ~ Charles Leiserson.&lt;/p&gt;
    &lt;/blockquote&gt;

    &lt;blockquote&gt;
      &lt;p&gt;Eat, sleep, and breathe a problem until you crack it. Become the world’s foremost expert on your thesis topic. Surpass your advisor.  ~ Daniel Jackson.&lt;/p&gt;
    &lt;/blockquote&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;http://www.ai.mit.edu/courses/6.899/papers/ted.htm&quot;&gt;On how to write papers, Ted Adelson&lt;/a&gt;.&lt;/p&gt;

    &lt;blockquote&gt;
      &lt;p&gt;Start by stating which problem you are addressing, keeping the audience in mind.  They must care about it, which means that sometimes you must tell them why they should care about the problem.  Then state briefly what the other solutions are to the problem, and why they aren’t satisfactory.  If they were satisfactory, you wouldn’t need to do the work.  Then explain your own solution, compare it with other solutions, and say why it’s bettter.  At the end, talk about related work where similar techniques and experiments have been used, but applied to a different problem.  Since I developed this formula, it seems that all the papers I’ve written have been accepted.&lt;/p&gt;
    &lt;/blockquote&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;vladen-koltuns-advice&quot;&gt;&lt;a href=&quot;https://www.cc.gatech.edu/~parikh/citizenofcvpr/static/slides/koltun_doing_good_research.pdf&quot;&gt;Vladen Koltun’s Advice&lt;/a&gt;&lt;/h3&gt;
&lt;ul&gt;
  &lt;li&gt;Picking a problem
    &lt;ul&gt;
      &lt;li&gt;Formulate a larger goal.&lt;/li&gt;
      &lt;li&gt;Personally meaningful.&lt;/li&gt;
      &lt;li&gt;Fits into the scheme of collective progress.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Analyze bottlenecks.&lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Understand the state of the art.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;Making a contribution:
    &lt;ul&gt;
      &lt;li&gt;Read the papers.&lt;/li&gt;
      &lt;li&gt;Look for unwarranted assumptions.&lt;/li&gt;
      &lt;li&gt;What are the limitations? When will this break? How could this be done better?&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Reimplement a state-of-the-art technique
    &lt;ul&gt;
      &lt;li&gt;Reproduce the results.&lt;/li&gt;
      &lt;li&gt;Then bombard it with controlled experiments.&lt;/li&gt;
      &lt;li&gt;Look for surprises, cracks that lead to deeper realizations.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Be on the lookout for interesting contributions.&lt;/li&gt;
  &lt;li&gt;Many important findings are not what the researchers set out to find
    &lt;ul&gt;
      &lt;li&gt;“Scheele happened upon chlorine while trying to isolate manganese; Claude Bernard planned experiments to characterize the destructive agent in sugar but instead discovered the glycogenic function of the liver; and so on.”
 ~ Ramón y Cajal, Letters to a Young Investigator&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Publications
    &lt;ul&gt;
      &lt;li&gt;Quality, not quantity.&lt;/li&gt;
      &lt;li&gt;Do not compromise on methodology or ethics.&lt;/li&gt;
      &lt;li&gt;Be willing to bury drafts and move on.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Publication portfolio
    &lt;ul&gt;
      &lt;li&gt;Most are prosaic.&lt;/li&gt;
      &lt;li&gt;Some are significant.&lt;/li&gt;
      &lt;li&gt;None are sloppy.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;High standards
    &lt;ul&gt;
      &lt;li&gt;Bury the weak, boring, and sloppy results.&lt;/li&gt;
      &lt;li&gt;Weak and sloppy work is a drain on the community. Can mislead. Goes against the goal of contributing something useful to the community.&lt;/li&gt;
      &lt;li&gt;Quantity is easy. The community doesn’t need more quantity.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Research over time
    &lt;ul&gt;
      &lt;li&gt;Research begets research.&lt;/li&gt;
      &lt;li&gt;Keep track of favorite problems, revisit occassionally.&lt;/li&gt;
      &lt;li&gt;Go back to the larger goals.&lt;/li&gt;
      &lt;li&gt;Read. A lot.&lt;/li&gt;
      &lt;li&gt;Write down ideas. Talk to people.&lt;/li&gt;
      &lt;li&gt;Quiet time for reading, writing, thinking.&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;For pete’s sake, get a good work ethic
    &lt;ul&gt;
      &lt;li&gt;
        &lt;p&gt;I do not believe a person can ever leave their business. They ought to think of it by day and dream of it by night. […] if they intend to go forward and do anything, the whistle is only a signal to start thinking over the day’s work in order to discover how it might be done better. […] The person who has the largest capacity for work and thought is the person who is bound to succeed.
  ~ Henry Ford, My Life and Work.&lt;/p&gt;
      &lt;/li&gt;
      &lt;li&gt;
        &lt;p&gt;In science as in the lottery, luck favors those who wager the most – that is, by another analogy, those who are tilling constantly the ground in their garden.
    ~ Ramón y Cajal, Letters to a Young Investigator.&lt;/p&gt;
      &lt;/li&gt;
      &lt;li&gt;
        &lt;p&gt;Successful people exhibit more activity, more energy, than most people do. They look more places, they work harder, they think longer than less successful people. Knowledge and ability are much like compound interest – the more you do the more you can do, and the more opportunities are open for you.
   ~ Hamming, Striving for Greatness in All You Do.&lt;/p&gt;
      &lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;david-mermin&quot;&gt;&lt;a href=&quot;http://www.ai.mit.edu/courses/6.899/papers/mermin.pdf&quot;&gt;David Mermin&lt;/a&gt;.&lt;/h3&gt;

&lt;p&gt;Always punctuate your equations. Math is prose. Number all equations in your text. It helps your readers.&lt;/p&gt;

&lt;h3 id=&quot;more-advice&quot;&gt;More Advice.&lt;/h3&gt;

&lt;h4 id=&quot;daniel-liberzon-research-quotes&quot;&gt;&lt;a href=&quot;http://liberzon.csl.illinois.edu/quote-research.html&quot;&gt;Daniel Liberzon Research Quotes&lt;/a&gt;.&lt;/h4&gt;

&lt;h4 id=&quot;how-to-write-mathematics&quot;&gt;&lt;a href=&quot;https://sites.math.washington.edu/~lind/Resources/Halmos.pdf&quot;&gt;How to write Mathematics&lt;/a&gt;.&lt;/h4&gt;
&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;+ [Alternative downloads website](downloads/Halmos.pdf)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;daniel-liberzon---how-to-write-a-good-paper&quot;&gt;&lt;a href=&quot;http://liberzon.csl.illinois.edu/writing-guidelines.html&quot;&gt;Daniel Liberzon - How to write a good paper&lt;/a&gt;.&lt;/h4&gt;

&lt;h4 id=&quot;daniel-liberzon---how-to-peer-review&quot;&gt;&lt;a href=&quot;http://liberzon.csl.illinois.edu/peer-review.pdf&quot;&gt;Daniel Liberzon - How to peer review&lt;/a&gt;.&lt;/h4&gt;

&lt;h4 id=&quot;george-m-whitesides--writing-a-paper&quot;&gt;&lt;a href=&quot;https://onlinelibrary.wiley.com/doi/pdf/10.1002/adma.200400767&quot;&gt;George M. Whitesides – Writing a Paper&lt;/a&gt;.&lt;/h4&gt;

&lt;h4 id=&quot;dmitri-bertsekas--ten-simple-rules-for-mathematical-writing&quot;&gt;&lt;a href=&quot;http://newslab.ece.ohio-state.edu/for%20students/resources/tenrules.pdf&quot;&gt;Dmitri Bertsekas – Ten Simple Rules for Mathematical Writing&lt;/a&gt;.&lt;/h4&gt;

&lt;h4 id=&quot;don-knuth-mathematical-writing&quot;&gt;&lt;a href=&quot;http://jmlr.csail.mit.edu/reviewing-papers/knuth_mathematical_writing.pdf&quot;&gt;Don Knuth –Mathematical Writing&lt;/a&gt;.&lt;/h4&gt;

&lt;h4 id=&quot;don-knuth--the-elements-of-mathematical-writing&quot;&gt;&lt;a href=&quot;https://www.mendeley.com/viewer/?fileId=36fa79c8-f907-0861-18b9-db563e2ef45f&amp;amp;documentId=7933d91e-f248-3103-9132-4657a82411f2&quot;&gt;Don Knuth – The Elements of Mathematical Writing&lt;/a&gt;.&lt;/h4&gt;

&lt;h4 id=&quot;n-david-mermin--whats-wrong-with-these-equations&quot;&gt;&lt;a href=&quot;https://www.mendeley.com/viewer/?fileId=6a8199f8-2d25-e7ef-28ac-579137975e93&amp;amp;documentId=779c442f-635a-3d6a-b64f-7ee8315d8aa2&quot;&gt;N. David Mermin – What’s Wrong With These Equations&lt;/a&gt;.&lt;/h4&gt;
</description>
        <pubDate>Thu, 02 Aug 2018 14:21:00 +0000</pubDate>
        <link>http://scriptedonachip.com/good-research</link>
        <guid isPermaLink="true">http://scriptedonachip.com/good-research</guid>
        
        
        <category>research</category>
        
        <category>good-research</category>
        
      </item>
    
      <item>
        <title>What&apos;s behind IEEE RAS Best Conference Papers?</title>
        <description>&lt;p&gt;Through the looking-glass, tired of working my brain out while familiarizing myself with a giant codebase I was reverse-engineering to prove a theory for an upcoming conference, I started asking myself why I was doing what I was doing? Just to get a paper out? Or to make a great contribution to science and my field? I googled something along the lines of “&lt;em&gt;How to write a best IEEE RAS conference paper&lt;/em&gt;”. What I found was very interesting as I came across this &lt;a href=&quot;/assets/ieee-best-paper/best_paper.pdf&quot;&gt;IEEE Student Activities Committee Paper&lt;/a&gt;. It offers
great insight into what constitutes writing a paper that merits IEEE Robotics and Automation Society best conference awards.&lt;/p&gt;

&lt;p&gt;I hope you enjoy reading it as much as I did.&lt;/p&gt;

</description>
        <pubDate>Sat, 24 Jun 2017 09:15:00 +0000</pubDate>
        <link>http://scriptedonachip.com/ieee-best-papers</link>
        <guid isPermaLink="true">http://scriptedonachip.com/ieee-best-papers</guid>
        
        
        <category>best-paper,</category>
        
        <category>IEEE,</category>
        
        <category>RAS</category>
        
      </item>
    
      <item>
        <title>On the necessary and sufficient conditions for optimal controllers</title>
        <description>
&lt;script type=&quot;text/x-mathjax-config&quot;&gt;
MathJax.Hub.Config({
  TeX: { equationNumbers: { autoNumber: &quot;AMS&quot; } }
});
&lt;/script&gt;

&lt;!--Mathjax Parser --&gt;
&lt;script type=&quot;text/javascript&quot; async=&quot;&quot; src=&quot;https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML&quot;&gt;
&lt;/script&gt;

&lt;script type=&quot;text/x-mathjax-config&quot;&gt;
MathJax.Hub.Config({
  tex2jax: {inlineMath: [[&apos;$&apos;,&apos;$&apos;], [&apos;\\(&apos;,&apos;\\)&apos;]]}
});
&lt;/script&gt;

&lt;!-- ### &lt;center&gt;Optimal Controllers: &lt;/center&gt; --&gt;
&lt;p&gt;This post deals with understanding the necessary and sufficient conditions, fundamental Lipschitz continuity assumptions and the terminal boundary conditions imposed on the Hamilton-Jacobi equation to assure that the problem of minimizing an integral performance index is well-posed.&lt;/p&gt;

&lt;h4 id=&quot;problem-statement&quot;&gt;Problem Statement&lt;/h4&gt;

&lt;p&gt;Suppose we have the following nonlinear dynamical system&lt;/p&gt;

&lt;p&gt;\begin{equation}            \label{eq:system}
  \dot{x}  =f(x, u, t), \qquad \qquad x(t_0) = x_0
\end{equation}&lt;/p&gt;

&lt;p&gt;which starts at state, \(x_0\) and time, \(t_0\).&lt;/p&gt;

&lt;h5 id=&quot;assumption-i&quot;&gt;&lt;strong&gt;Assumption I&lt;/strong&gt;&lt;/h5&gt;

&lt;p&gt;If the function \(f(\centerdot)\) is
continuously differentiable in all its arguments, then the initial value problem (IVP) of \eqref{eq:system} has a &lt;u&gt;unique solution&lt;/u&gt; on a finite time interval; this is a sufficient assumption (Khalil, 1976).&lt;/p&gt;

&lt;h5 id=&quot;assumption-ii&quot;&gt;&lt;strong&gt;Assumption II&lt;/strong&gt;&lt;/h5&gt;

&lt;p&gt;\(T\) is sufficiently small enough to reside within the time interval where the system’s solutions are defined.&lt;/p&gt;

&lt;p&gt;Qualitatively, our goal is to &lt;strong&gt;optimally&lt;/strong&gt; control the system when it starts in a state \(x_0\), at time \(t_0\), to a neighborhood of the terminal manifold \(T\), whilst exerting as minimal a control energy as possible. Quantitatively, we can define this goal in terms of an index of performance evaluation defined thus:&lt;/p&gt;

&lt;p&gt;\begin{equation}            \label{eq:cost}
  J = J(x(t_0, u(\centerdot), t_0)  = \int\limits_{t=t_0}^{T} L\left(x(\tau), u(\tau), \tau\right)d\tau + V(x(T))
\end{equation}&lt;/p&gt;

&lt;p&gt;where \(J\) is evaluated along the trajectories of the system \(x(t)\), based on an applied control \(u(\centerdot)|_{t_0 \le t \le T} \).
With \(L\left(x(\tau), u(\tau), \tau\right)\) as the instantaneous cost and \(V(x(T))\) as the terminal cost (which are nonnegative funtions of their arguments), we can think of \(J\) as the total amount of actions we take (controls) and the state energy utilized in bearing the states from \(x_0\) to a neighborhood of the terminal manifold \(V(x(T)) = 0\).&lt;/p&gt;

&lt;p&gt;The question to ask then is that given the cost of performance index \(J\), how do we find a control law \(u^\star\) that is optimal along a unique state trajectory, \(x^\star\), in the interval \([t_0, T]\)? This optimal cost would be the minimum of all the possible costs that we could possibly incur when we implement the optimal control law \(u^\star\). Mathematically, we can express this cost as:&lt;/p&gt;

&lt;p&gt;\begin{gather}
  J^\star(x(t_0), t_0)  = \int\limits_{t=t_0}^{T} L \left(x^{\star}(\tau), u^\star(\tau), \tau \right) d\tau + V(x^\star(T)) &lt;br /&gt;
                 = \min_{ u_{[t_0, T]}} J(x_0, u, t_0)
\end{gather}&lt;/p&gt;

&lt;p&gt;Therefore, the optimal cost is a function of the starting state and time so that we can write:&lt;/p&gt;

&lt;p&gt;\begin{equation}
  J^\star(x(t_0), t_0) = \min_{ u_{[t_0, T]}} J(x(t_0), u(\centerdot), t_0) = \min_{ u_{[t_0, T]}} \int\limits_{t_0}^{T} L\left(x(\tau), u(\tau), \tau\right)d\tau + V(x(T))
\end{equation}&lt;/p&gt;

&lt;p&gt;Now, assume that we start at an arbitrary initial condition \(x\), at time \(t\), it follows that the optimal cost-to-go from \(x(t)\) to \(x(T)\) is (abusing notation and dropping the templated arguments in \(J\)):&lt;/p&gt;

&lt;p&gt;\begin{equation}          \label{eq:cost-to-go}
  J^\star(x, t) = \min_{ u_{[t, T]}} \left[\int\limits_{t}^{T} L\left(x(\tau), u(\tau), \tau\right)d\tau\right] + V(x(T))
\end{equation}&lt;/p&gt;

&lt;p&gt;Things get a little bit interesting when we splice up the integral kernel in \eqref{eq:cost-to-go} along two different time-paths, namely:&lt;/p&gt;

&lt;p&gt;\begin{equation}        \label{eq:spliced}
  J^\star(x, t) = \min_{ u_{[t, T]}} \left[\int\limits_{t}^{t_1} L\left(x(\tau), u(\tau), \tau\right)d\tau + \int\limits_{t_1}^{t_2} L\left(x(\tau), u(\tau), \tau\right)d\tau\right] + V(x(T))
\end{equation}&lt;/p&gt;

&lt;p&gt;We can split the minimization over two time intervals, e.g.,&lt;/p&gt;

&lt;p&gt;\begin{equation}        \label{eq:two_mins}
  J^\star(x, t) = \min_{ u_{[t, t_1]}} \min_{ u_{[t_1, t_2]}} \left[\int\limits_{t}^{t_1} L\left(x(\tau), u(\tau), \tau\right)d\tau + \int\limits_{t_1}^{t_2} L\left(x(\tau), u(\tau), \tau\right)d\tau\right] + V(x(T))
\end{equation}&lt;/p&gt;

&lt;p&gt;Equation \eqref{eq:two_mins} gives the beautiful intuition that one can divide the integration into two or more time slices, solve the optimal control problem for each time slice and in the overall, minimize the effective cost function \(J\) of the overall system. This in essence is a statement of &lt;a href=&quot;https://en.wikipedia.org/wiki/Richard_E._Bellman&quot;&gt;Richard E. Bellman&lt;/a&gt;’s principle of optimality:&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;Bellman’s Principle of Optimality: An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;blockquote&gt;
  &lt;p&gt;– Bellman, Richard. Dynamic Programming, 1957, Chap. III.3.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;With the principle of optimality, the problem takes a more intuitive meaning, namely that the cost to go from \(x\) at time \(t\) to a terminal state \(x(T)\) can be computed by minimizing the sum of the cost to go from \(x = x(t)\) to \(x_1 = x(t_1)\) and then, the  optimal cost-to-go from \(x_1\) onwards.&lt;/p&gt;

&lt;p&gt;Therefore, \eqref{eq:two_mins} can be restated as:&lt;/p&gt;

&lt;p&gt;\begin{equation}        \label{eq:two_mins_sep}
  J^\star(x, t) = \min_{ u_{[t, t_1]}} \left[\int\limits_{t}^{t_1} L\left(x(\tau), u(\tau), \tau\right)d\tau +  \underbrace{\min_{ u_{[t_1, t_2]}} \int\limits_{t_1}^{t_2} L\left(x(\tau), u(\tau), \tau\right)d\tau + V(x(T))}_{J^\star(x_1, \, t_1)} \right]
\end{equation}&lt;/p&gt;

&lt;p&gt;\(J^\star(x_1, \, t_1)\) in \eqref{eq:two_mins_sep} can be seen as the optimal cost-to-go from \(x_1\) to \(x(T)\), with the overall cost given by&lt;/p&gt;

&lt;p&gt;\begin{equation}         \label{eq:optimal_pre}
  J^\star(x, t) = \min_{ u_{[t, t_1]}} \left[\int\limits_{t}^{t_1} L\left(x(\tau), u(\tau), \tau\right)d\tau +  J^\star(x_1, \, t_1)  \right]
\end{equation}&lt;/p&gt;

&lt;p&gt;Replacing \(t_1\) by \(t + \delta t\) and with the assumption that \(J^\star(x, t)\) is differentiable, we can expand \eqref{eq:optimal_pre} into a first-order Taylor series around \((\delta t, x)\) as follows:&lt;/p&gt;

&lt;p&gt;\begin{equation}        \label{eq:taylor}
  J^\star(x, t) = \min_{ u_{[t, t + \delta]}} \left[ L\left(x, u, \tau\right)\delta t  +  J^\star(x, \, t) + \left(\dfrac{\partial J^\star(x, t)}{\partial t}\right) \delta t + \left(\dfrac{\partial J^\star(x, t)}{\partial x} \right) \delta x + o(\delta)  \right]
\end{equation}&lt;/p&gt;

&lt;p&gt;where \(o(\delta)\) denotes higher order terms satisfying \(\lim_{\delta \rightarrow 0}\dfrac{o(\delta)}{\delta} = 0\).&lt;/p&gt;

&lt;p&gt;Refactoring \eqref{eq:taylor}, we find that&lt;/p&gt;

&lt;p&gt;\begin{equation}        \label{eq:hamiltonian_pre}
  \dfrac{\partial J^\star(x, t)}{\partial t}  = -\min_{ u_{[t, t + \delta]}} \left[ L\left(x, u, \tau\right)  +   \left(\dfrac{\partial J^\star(x, t)}{\partial x} \right) \underbrace{\dot{x}(\centerdot)}_{f(x,u,t)}  \right]
\end{equation}&lt;/p&gt;

&lt;p&gt;We shall define the components in the square column of the above equation as the &lt;strong&gt;Hamiltonian&lt;/strong&gt;, \(H(\centerdot)\) such that  \eqref{eq:hamiltonian_pre} can be thus rewritten:&lt;/p&gt;

&lt;p&gt;\begin{equation}        \label{eq:hamiltonian}
  \dfrac{\partial J^\star(x, t)}{\partial t}  = -\min_{ u_{[t, t + \delta]}} H\left(x, \nabla_x J^\star (x, t), u, t \right)
\end{equation}&lt;/p&gt;

&lt;p&gt;Based on the smoothness assumption of all function arguments in \eqref{eq:system},
when the linear sensitivity of the Hamiltonian to changes in \(u\) is zero, then
\(\nabla H_u\) &lt;strong&gt;must&lt;/strong&gt; vanish at the optimal point i.e.,&lt;/p&gt;

&lt;p&gt;\begin{equation}      \label{eq:hamiltonian_deri}
  \nabla H_u(x, \nabla_x J^\star (x, t), u, t) = 0
\end{equation}&lt;/p&gt;

&lt;p&gt;ensuring that we satisfy the &lt;strong&gt;local optimality&lt;/strong&gt; property of the controller. In addition, if the Hessian of the Hamiltonian is positive definite along the trajectories of the solution, i.e.,&lt;/p&gt;

&lt;p&gt;\begin{equation} &lt;br /&gt;
  \dfrac{\partial^2 H}{\partial^2 u} &amp;gt; 0
\end{equation}&lt;/p&gt;

&lt;p&gt;then we have the sufficient condition for global optimality. These conditions are referred to as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Legendre%E2%80%93Clebsch_condition&quot;&gt;Legendre-Clebsch&lt;/a&gt; conditions, essentially guaranteeing that over a singular arc, the Hamiltonian is minimized.&lt;/p&gt;

&lt;p&gt;You begin to see the beauty of optimal control in that \eqref{eq:hamiltonian_deri} allows us to translate the complicated functional minimization integral of \eqref{eq:cost} into a minimization problem that can be solved by ordinary calculus.&lt;/p&gt;

&lt;p&gt;If we let&lt;/p&gt;

&lt;p&gt;\begin{equation}    &lt;br /&gt;
  H^\star(x, \nabla_x J^\star (x, t), t) = \min_u \left[H(x, \nabla_x J^\star (x, t), u, t)\right]
\end{equation}&lt;/p&gt;

&lt;p&gt;then it follows that solving \eqref{eq:hamiltonian_deri} for the optimal \(u = u^\star\) and putting the result in \eqref{eq:hamiltonian}, one obtains the &lt;em&gt;&lt;strong&gt;Hamilton-Jacobi-Bellman&lt;/strong&gt;&lt;/em&gt; pde whose solution is the optimal cost \(J^\star(x(t), t)\) such that&lt;/p&gt;

&lt;p&gt;\begin{equation}        \label{eq:optimal_cost}
  \dfrac{\partial J^\star(x, t)}{\partial t}  = -H^\star \left(x, \nabla_x J^\star (x, t), u, t \right)
\end{equation}&lt;/p&gt;

&lt;p&gt;We can introduce a boundary condition that assures that the cost function of \eqref{eq:cost} is well-posed viz,&lt;/p&gt;

&lt;p&gt;\begin{equation}        \label{eq:boundary_cost}
  J^\star(x(T), T)  = V(x(T))
\end{equation}&lt;/p&gt;

&lt;p&gt;Taken together, equations \eqref{eq:optimal_cost} allows us to analytically solve for the instanteneous &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;kinetic energy&lt;/code&gt; of the cost function in \eqref{eq:cost} and \eqref{eq:boundary_cost}  allows us to solve for the boundary condition that assure the sufficiency of an optimal control law to exist. If we can solve for \(u^\star\) from \(J^\star(x,t)\), then \eqref{eq:boundary_cost} must constitute the optimal control policy for the nonlinear dynamical system in \eqref{eq:system} given the cost index \eqref{eq:cost}.&lt;/p&gt;

&lt;h3 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h3&gt;

&lt;p&gt;Notice that the optimal policy \(u^\star(t)\) is basically an open-loop control strategy. Why so? \(u^\star\) was derived as a function of time \(t\). As a result, the strategy may not be robust to uncertainties and may be very sensitive. For practical applications, we generally want to have a feedback control policy that is state dependent in order to guarantee robustness to parametric variations and achieve robust stability and performance. Such a \(u = u^\star(x)\) would be helpful in analyzing the stability of states and convergence of system dynamics to equilibrium for all future times. Will post such methods in the future.&lt;/p&gt;

&lt;h3 id=&quot;summary&quot;&gt;Summary&lt;/h3&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th style=&quot;text-align: left&quot;&gt;Properties&lt;/th&gt;
      &lt;th style=&quot;text-align: right&quot;&gt;Equations&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;Dynamics:&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;\(\dot{x}  =f(x, u, t), \quad x(t_0) = x_0 \)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;Cost:&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;\(J(x,u,\tau) = \int\limits_{t=t_0}^{T} L\left(x(\tau), u(\tau), \tau\right)d\tau + V(x(T))\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;Optimal cost :&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;\(J^\star(x,t) = \min_{u[t,T]}J\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;Hamiltonian:&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;\(H(x,u,t) =  L\left(x, u, \tau\right)  +   \left(\dfrac{\partial J^\star(x, t)}{\partial x} \right) f(x,u,t)\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;Optimal Control:&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;\(u^\star(t) =  H^\star(x,u,t) =  \nabla H_u(x,u,t)\)&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td style=&quot;text-align: left&quot;&gt;HJB Equation:&lt;/td&gt;
      &lt;td style=&quot;text-align: right&quot;&gt;\(-\dfrac{\partial J^\star(x,t)}{\partial t} = H^\star(x, \nabla_x J^\star(x,t),t) )\)  and   \(J^\star(x(T), T)  = V(x(T))\)&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;h3 id=&quot;further-readings&quot;&gt;Further Readings&lt;/h3&gt;

&lt;p&gt;&lt;a href=&quot;https://web.archive.org/web/20050110161049/http://www.wu-wien.ac.at/usr/h99c/h9951826/bellman_dynprog.pdf&quot;&gt;Richard Bellman: On The Birth Of Dynamic Programming&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://www.amazon.com/Optimal-Control-Quadratic-Methods-Engineering/dp/0486457664&quot;&gt;Optimal Control: Linear Quadratic Methods&lt;/a&gt;&lt;/p&gt;

</description>
        <pubDate>Sun, 04 Jun 2017 13:28:00 +0000</pubDate>
        <link>http://scriptedonachip.com/optimal-control</link>
        <guid isPermaLink="true">http://scriptedonachip.com/optimal-control</guid>
        
        
        <category>control</category>
        
        <category>optimal-control</category>
        
      </item>
    
      <item>
        <title>Should I use ROS or MuJoCo?</title>
        <description>&lt;p&gt;This was my answer to a question posted in an email thread to our research group’s email lists. The question goes like this:&lt;/p&gt;

&lt;p&gt;&lt;br /&gt;
&lt;br /&gt;
&lt;strong&gt;QUESTION&lt;/strong&gt;
&lt;br /&gt;
&lt;strong&gt;__&lt;/strong&gt;&lt;strong&gt;__&lt;/strong&gt;&lt;strong&gt;__&lt;/strong&gt;&lt;strong&gt;__&lt;/strong&gt;&lt;strong&gt;__&lt;/strong&gt;&lt;strong&gt;__&lt;/strong&gt;&lt;em&gt;__&lt;/em&gt;&lt;br /&gt;
From: XXX@uni-x.edu &lt;br /&gt;
Sent: Sunday, May 14, 2017 9:29 AM &lt;br /&gt;
To: XXX@lists.uni-x.edu &lt;br /&gt;
Subject: RE: [robotec] MuJoCo &lt;br /&gt;&lt;/p&gt;

&lt;p&gt;From the documentation, looks like MuJoCo is faster and therefore better to simulate computational intensive controllers like MPC. Gazebo provides other engines as well and seems like is more popular in the ROS community. How to choose between the available options? Which one would you recommend?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br /&gt;
XXX&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Answer&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;TL; DR:&lt;/strong&gt;
If you do not care for accuracy of simulated controller numerical results, if you are not simulating parallel linkages, if you do not need code parallelization (i.e. your computation is not crazy intensive) or if you are not simulating contact and friction, I would choose ROS. Easy to use and straightforward to build fairly complex models.&lt;/p&gt;

&lt;h1 id=&quot;proper-long-answer&quot;&gt;&lt;strong&gt;Proper (Long) Answer&lt;/strong&gt;&lt;/h1&gt;
&lt;p&gt;ROS and Gazebo (OSRF tools) are indeed popular in the robotics community like you mentioned and they have their pros. It took me a while to see their limit when using them for research purposes.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Definition&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;ROS = Plumbing + Tools + Capabilities + Ecosystem&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Plumbing&lt;/code&gt;: ROS provides publish-subscribe messaging infrastructure designed to support the quick and easy construction of distributed computing systems.&lt;/p&gt;

&lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Tools&lt;/code&gt;: ROS provides an extensive set of tools for configuring, starting, introspecting, debugging, visualizing, logging, testing, and stopping distributed computing systems.&lt;/p&gt;

&lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Capabilities&lt;/code&gt;: ROS provides a broad collection of libraries that implement useful robot functionality, with a focus on mobility, manipulation, and perception.&lt;/p&gt;

&lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Ecosystem&lt;/code&gt;: ROS is supported and improved by a large community, with a strong focus on integration and documentation. ros.org is a one-stop-shop for finding and learning about the thousands of ROS packages that are available from developers around the world. answers.ros.org is a rich online community of ros packages users from around the world asking questions and getting help on how to use ROS.&lt;/p&gt;

&lt;p&gt;So getting a simple dynamics kicking should not be a lot of hassle as the documentation is rich and the online community is very active in supporting newbies.&lt;/p&gt;

&lt;p&gt;In the early days, the plumbing, tools, and capabilities were tightly coupled, which has both advantages and disadvantages. On the one hand, by making strong assumptions about how a particular component will be used, developers are able to quickly and easily build and test complex integrated systems. On the other hand, users are given an “all or nothing” choice: to use an interesting ROS component, you pretty much had to jump in to using all of ROS.&lt;/p&gt;

&lt;p&gt;8+ years in after Andrew Ng and co. conceived the platform, the core system has matured considerably, and developers are hard at work refactoring code to separate plumbing from tools from capabilities, so that each may be used in isolation. In particular, people are aiming for important libraries that were developed within ROS to become available to non-ROS users in a minimal-dependency fashion (e.g. OMPL and PCL libraries).&lt;/p&gt;

&lt;p&gt;Disclaimer: Borrowed  from Brian Gerkey’s/my answer to a similar quora question about a year ago.&lt;/p&gt;

&lt;p&gt;For serially linked robot arms and other non-parallel linkages, ROS is a great simulation tool and “middleware”.  However, there are bottlenecks with ROS.&lt;/p&gt;

&lt;p&gt;What ROS calls URDF (Universal Robot Description Format), which is the abstraction tool for rigid body dynamics, is not universal in any sense of the word. URDF models written in ROS are  out-of-the-box incompatible with Gazebo, its sister physics engine (see this &lt;a href=&quot;http://answers.gazebosim.org/question/14891/conversion-from-urdf-to-sdf-using-gzsdf-issues/&quot;&gt;question/wiki&lt;/a&gt;).  More so, state representation in OSRF tools such as ROS is represented in a tree-like manner. I learnt this late last year when simulating parallel linkages. The internal ROS XML  parser interprets constructed linkages as a deep binary tree and not graphs. This makes  simulating parallel linkages almost (actually) impossible. Repeat, actually impossible. They have a fix for this in Gazebo SDF but it is not straightforward. So developers spend a huge chunk of time migrating code from one OSRF framework to another.&lt;/p&gt;

&lt;p&gt;Good controller algorithm formulations are based on numerical optimization (think MPC, differential dynamic programming, sampling-based motion-planning or reinforcement learning). Gazebo was designed around the ODE (Open Dynamics Engine) and Bullet physics engines which provide the states in over-complete Cartesian coordinates and enforce joint constraints via numerical optimization. This is good enough for disconnected bodies with few joint constraints but becomes a pain for complex dynamics such as humanoids or simulating human-robot interactions. Running complex simulations for huge candidate evaluations of humanoids can run into months using ROS (e.g. Todorov’s de novo synthesis). Whereas MUJOCO is optimized for parallel processing, distributed evaluation of possible controllers from which a candidate controller is chosen.&lt;/p&gt;

&lt;p&gt;ODE simulators optimize the controller to the engine. This makes the controller cheat during simulations in ways that mean generated control laws may be physically unrealizable. Speed and accuracy? Controller optimization with MoveIt! (a motion planning framework from OSRF) is mostly done in a single threaded code without the advantage of explicit parallelization of code to make e.g. IK solutions faster. Implementation of concurrency and multithreading is left to the user (this is a big no-no for someone not interested in software engineering).&lt;/p&gt;

&lt;p&gt;ROS is strictly written based on the assumption that the user is running a Linux kernel. So users not familiar with Linux are thrown aback when they first get exposed to it. With MUJOCO, you do not need Linux or OSX as it works on Windows OS just fine. MUJOCO also use an XML parser to interpret links and joints and so it is able to read ROS URDFs and xacro files okay. But it doesn’t work the other way (see &lt;a href=&quot;http://www.mujoco.org/forum/index.php?threads/ros-gazebo-integration.3371/&quot;&gt;this answer&lt;/a&gt; from Todorov)&lt;/p&gt;

&lt;p&gt;MPC implementations are elegant only when the model is accurate. Unexpected poor performance of an MPC controller will often be due to poor modeling assumptions (Rossiter).  If the simulation engine emphasizes simulation stability over control law precision, we have a problem.  And this is my problem with Gazebo and ROS generally. I read it somewhere in one of Todorov’s papers (can’t remember where I found it) that the floating point ops of MUJOCO were unit tested to &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;&amp;gt;&amp;gt;355&lt;/code&gt; decimal points. The OSRF community may be good at community based software engineering for robotics but you have to give Todorov the credit. He had the patience and tenacity to develop such a robust software for control simulation. People stopped paying attention to floating point operational precision back in the late 80’s/90’s.&lt;/p&gt;

&lt;p&gt;What’s more? MUJOCO allows you to write your models in C. Engineers are head over boots for Matlab but I am all for a program or modeling software that stays close to ones and zeros as much as possible. It means being less dumbfounded when things do not work as you envisioned and greater flexibility in being the master and architect of your creation.&lt;/p&gt;
</description>
        <pubDate>Sun, 14 May 2017 13:28:00 +0000</pubDate>
        <link>http://scriptedonachip.com/mujoco-ros</link>
        <guid isPermaLink="true">http://scriptedonachip.com/mujoco-ros</guid>
        
        
        <category>Q&amp;A</category>
        
        <category>ros</category>
        
        <category>mujoco</category>
        
      </item>
    
  </channel>
</rss>
