TY - JOUR

T1 - Stochastic optimal control as non-equilibrium statistical mechanics

T2 - Calculus of variations over density and current

AU - Chernyak, Vladimir Y.

AU - Chertkov, Michael

AU - Bierkens, Joris

AU - Kappen, Hilbert J.

PY - 2014/1/17

Y1 - 2014/1/17

N2 - In stochastic optimal control (SOC) one minimizes the average cost-to-go, that consists of the cost-of-control (amount of efforts), cost-of-space (where one wants the system to be) and the target cost (where one wants the system to arrive), for a system participating in forced and controlled Langevin dynamics. We extend the SOC problem by introducing an additional cost-of-dynamics, characterized by a vector potential. We propose derivation of the generalized gauge-invariant Hamilton-Jacobi-Bellman equation as a variation over density and current, suggest hydrodynamic interpretation and discuss examples, e.g., ergodic control of a particle-within-a-circle, illustrating non-equilibrium space-time complexity.

AB - In stochastic optimal control (SOC) one minimizes the average cost-to-go, that consists of the cost-of-control (amount of efforts), cost-of-space (where one wants the system to be) and the target cost (where one wants the system to arrive), for a system participating in forced and controlled Langevin dynamics. We extend the SOC problem by introducing an additional cost-of-dynamics, characterized by a vector potential. We propose derivation of the generalized gauge-invariant Hamilton-Jacobi-Bellman equation as a variation over density and current, suggest hydrodynamic interpretation and discuss examples, e.g., ergodic control of a particle-within-a-circle, illustrating non-equilibrium space-time complexity.

KW - Bellman-Hamilton-Jacobi equation

KW - gauge transformations

KW - non-equilibrium statistical physics

KW - stochastic optimal control

UR - http://www.scopus.com/inward/record.url?scp=84891349702&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84891349702&partnerID=8YFLogxK

U2 - 10.1088/1751-8113/47/2/022001

DO - 10.1088/1751-8113/47/2/022001

M3 - Article

AN - SCOPUS:84891349702

VL - 47

JO - Journal of Physics A. Mathematical and Theoretical

JF - Journal of Physics A. Mathematical and Theoretical

SN - 1751-8113

IS - 2

M1 - 022001

ER -