Postprint version. Published in Stochastic Processes and their Applications, Volume 24, Issue 1, February 1, 1987, pages 61-76. Copyright © 1987 Elsevier. The definitive version is available at http://dx.doi.org/10.1016/0304-4149(87)90028-7.
NOTE: At the time of publication, the author Theodore P. Hill was not yet affiliated with Cal Poly.
For countable-state decision processes (dynamic programming problems), a general class of objective functions is identified for which it is shown that good Markov strategies always exist. This class includes product and lim inf rewards, as well as practically all the classical dynamic programming expected payoff functions.