Oregon State University

Calendar

Calendars

Event Details

MS Final Examination – Thai Duong


Monday, June 10, 2013 2:00 PM - 4:00 PM

Adiabatic Markov Decision Process: Convergence of Value Iteration Algorithm
Markov Decision Process (MDP) is a well-known framework for devising the optimal decision making strategies under uncertainty. Typically, the decision maker assumes a stationary environment which is characterized by a time-invariant transition probability matrix.  However, in many real-world scenarios, this assumption is not justified, thus the optimal strategy might not provide the expected performance. In this thesis, we study the performance of the classic Value Iteration (VI) algorithm for solving an MDP problem under non-stationary environments. Specifically, the non-stationary environment is modeled as a sequence of time-variant transition probability matrices governed by an adiabatic evolution inspired from quantum mechanics.

We characterize the performance of the VI algorithm subject to the rate of change of the underlying environment. The performance is measured in terms of the convergence rate to the optimal average reward. We show two examples of queuing systems that make use of our analysis framework.

Major Advisor: Thinh Nguyen
Committee: Bella Bose
Committee: Prasad Tadepalli
Committee: Raviv Raich
GCR: Yevgeniy Kovchegov


Kelley Engineering Center (campus map)
1114
Nicole Thompson
1 541 737 3617
Nicole.Thompson at oregonstate.edu
Sch Elect Engr/Comp Sci
This event appears on the following calendars: