Discrete Time Stochastic Control and Dynamic Potential Games: The Euler Equation Approach

I was very conflicted about blogging this book at all, but I figured that given I have blogged all other non-fiction books this year so far I probably ought to at least talk a little bit about this one as well. I wrote this on goodreads:

“The book contained a brief review of some mathematics used in a couple of previous courses I’ve taken, with some new details added to the mix. Having worked with this stuff before is probably a requirement to get anything much out of it, as it is highly technical.”

Here are some observations/comments from the conclusion, providing a brief outline:

“In this book we have studied discrete-time stochastic optimal control problems (OCPs) and dynamic games by means of the Euler equation (EE) approach. […] In Chap. 2 we studied the EE approach to nonstationary OCPs in discrete-time. OCPs are usually solved by dynamic programming and the Lagrange method. The latter techniques for solving OCPs are based on iteration methods or rely on guessing the form of the value or the policy functions […] In contrast, the EE approach does not require an iteration method nor knowledge about the form of the value function; on the contrary, the value function can be computed after the OCP is solved. Following the EE approach, we have to solve a second-order difference equation (possibly nonlinear and/or nonhomogeneous); there are, however, many standard methods to do this. Both the EE […] and the transversality condition (TC) […] are known in the literature. The EE […] is typically deduced from the Bellman equation whereas the necessity of the TC […] is obtained by using approximation or perturbation results. Our main results in Chap. 2 require milder assumptions […] In Theorem 2.1 we obtain the EE (2.14) and the TC (2.15), as necessary conditions for optimality, using Gâteaux differentials. […] Chapter 3 was devoted to an inverse optimal problem in stochastic control. […] Finally, in Chap. 4, some results from Chaps. 2 and 3 were applied to dynamic games. Sufficient conditions to identify MNE [Markov–Nash equilibria] and OLNE [Open-loop Nash equilibria], by following the EE approach, were given […] one of our main objectives was to identify DPGs [Dynamic potential games] by generalizing the procedure of Dechert and O’Donnell for the SLG”

“Some advantages and shortcomings of the EE approach. A first advantage of using the EE to solve discrete-time OCPs is that it is very natural and straightforward, because it is an obvious extension of results on the properties of maxima (or minima) of differentiable functions. Indeed, as shown in Sect. 2.2, using Gâteaux differentials, the EE and some transversality condition are straightforward consequences of the elementary calculus approach. From our present point of view, the main advantage of the EE approach is that it allows us to analyze certain inverse OCPs required to characterize the dynamic potential games we are interested in. It is not clear to us that these inverse OCPs can be analyzed by other methods (e.g.,, dynamic programming or the maximum principle). On the other hand, a possible disadvantage is that the Euler equation might require some “guessing” to obtain a sequence that solves it. This feature, however, is common to other solution techniques such as dynamic programming.”

…

If none of the above makes much sense to you, I wouldn’t worry too much about it. Stuff like this, this, and this was covered in previous coursework of mine so I was familiar with some of the stuff covered in this book; stuff like this is part of what many economists learn during their education. I figured it’d be interesting to see a more ‘pure-math’ coverage of these things. It turned out, however, that many of the applications in the book are economics-related, so in a way the coverage was ‘less pure’ than I’d thought before I started out.

A couple of links I looked up along the way are these: Gâteaux derivative, Riccati equation, Borel set. I haven’t read this, but a brief google for some of the relevant terms above made that one pop up; it looks as if it may be a good resource if you’re curious to learn more about what this kind of stuff is about.