Abstract
The object is to find a control policy that minimizes the performance measure representing the total costs from cleaning up a contaminated aquifer by well extraction. The performance measure is quadratic in both the control and the state. However, a more generalized form used when finding the control, expanding its usability to any number of other applications. The minimization is subject to a dynamical system containing a Gauss Markov stochastic process. An additional stochastic process from the partially observed measurements requires the state variable to be approximated (analytically here) using a Kalman filter.
The system is discretized using the Crank-Nicholson Implicit Method. The control solution in the dynamical system is based on a variant of differential dynamic programming (DDP). It finds the control through a backwards sweep in time. The dynamical system is used to find the state during a forward time sweep. This method gives an approximate solution in part by perturbing the state and control with a small parameter. The final analytical solution involves the deterministic solution and the first two correction terms.
Several aspects of the stochastic DDP with perturbations are re-examined and refined. The perturbations are done systematically. All appropriate variables are expanded asymptotically and the Taylor approximations include all relevant terms. In addition, a particular form is not assumed for the solutions of variables or the cost-to-go.
A specific groundwater remediation application is used to investigate the method analytically and numerically. A notable item in the numerical procedure is the use of uni-directional coupling to avoid the nonlinearity of the difference equation used to find the contaminant concentration. The solution is a guide for setting the pumping rates used in groundwater remediation, as well as an indicator of the contamination level in the aquifer once remediation has begun.
