full^-1 @ rhs with a fixed (detached) LU factor, refined, and
differentiable through the residual instead of through the factor.
lu, pivots is the factor of full.detach(). It is applied as the
refined solve R of lu_solve_refined() (rounds rounds) and is
never differentiated. The dependence on full and rhs enters through
the residual rhs - full @ x of a detached start x = R(rhs), which is
corrected twice: x <- x + R(rhs - full @ x). The value is the refined
solution; the first derivative is the implicit one, A^-1 (db - dA x),
with A^-1 applied by the refined R; and the second unrolled
correction makes the second derivatives (Hessians, force-training
gradients of trainable parameters) right as well. With e = 1 - R A
the error of the factor, the value and both derivatives are accurate to
O(e^2). The backward costs two adjoint solves per correction,
O(N^2), where autograd through lu_factor costs O(N^3). Scriptable.
- Parameters:
-
- Return type:
Tensor
