Two approaches for optimization of discrete time Markovian decision processes are studied - the linear programming method and the policy iteration algoritm.