Advances in solving optimal control problems

Analytical methods for solving optimal control problems (OCPs) allow closed-form solutions to be obtained in low-dimensional conceptual models. A useful tool for studying qualitative phenomena, these reveal general patterns in the underlying processes of complex interactions in society, in the environment, and between them.

The widely used Pontryagin maximum principle (MP) supplies a set of necessary (and in some cases sufficient) conditions for optimality. In addition to state and control variables, such conditions have a so-called adjoint variable. This was originally seen as an auxiliary parameter. However, the properties of the adjoint variable play an important role when the MP is applied to solving OCPs, often allowing a single optimal process to be selected from a set of candidates derived from the core conditions of the MP. In economic applications the adjoint variable is used to represent the notion of the shadow price.

For many years the ASA program has contributed to advances in optimal control theory by developing special versions of the MP for OCPs on the infinite time horizon. These are the most common methodological tools for many applications, for example, economic growth theory.

In 2013 ASA made several new contributions. A class of infinite time-horizon OCPs with a dominating discount that appear in economic growth theory was studied [1]. An explicit formula for the adjoint variable was derived and it was shown that as well as satisfying core MP conditions, it also satisfied the additional stationarity condition for the Hamiltonian. The results obtained extend to a general case of non-differentiable value function.

There was consideration of how to interpret a solution to an OCP posed in terms of achieving a long-term goal from a perspective of short-term goals [2]. Using a stylized model of an enterprise as an illustrative example plus the MP as a solution tool, it was shown that by properly evaluating current shadow prices, a decision maker can reduce, or even eliminate, a conflict between long- and short-term interests in enterprise management. This effectively makes current short-term-optimal actions identical to long-term optimal actions.

Some properties of attainability sets of nonlinear dynamic systems were explored [3]; the results are useful for analyzing control systems with phase constraints.

In view of the growing complexity of socio-environmental problems considered by the scientific community, it is also important to develop numerical methods for solving OCPs that properly approximate an original solution and are computationally efficient.

In 2013 ASA continued its work on developing numerical approaches to solving OCPs on the infinite time horizon.  It is difficult to apply standard numerical algorithms to solve infinite time-horizon optimal control problems because the boundary condition on the right-hand side of the Hamiltonian system, which relates to the special properties of the adjoint variable, extends to infinity. 

There was a presentation and demonstration of the applicability of the new numerical method to localizing the search for optimal trajectories on the phase plane in the vicinity of the steady state of the Hamiltonian system by building a non-linear stabilizer [4] [5]. The non-linear stabilizer approach to developing a numerical algorithm for solving an infinite time-horizon optimal control problem with phase constraints was applied [6]. Scientists also addressed the problem of controlling a system affected by disturbing inputs and suggested an on-line state-reconstruction algorithm to build a feedback controller [7]. 

Last edited: 21 May 2014