Risk is a very important concept in many areas of every day life. Risk is the potential of gaining or losing something of value. The values include physical health, social status, emotional well-being or financial wealth, etc. Values can be gained or lost when taking risk resulting from a given action or inaction, foreseen or unforeseen. In real-life, many decisions are made to averse risks. For instance, most factories would choose to produce at a lower speed to avoid high risk of flawed products. Likewise, many people would not choose high-stake asset allocation portfolios with the possibility of high wins and losses. There are, however, other cases where risk-seeking actions are preferred, such as gambling. It is then of utmost importance to take risks into consideration when modelling a system.
Markov decision processes (MDPs) is a general mathematical framework to model a system. They are used in a wide area of disciplines, such as robotics, automated control, economics, and manufacturing. In many applications modelled by MDPs, it is crucial to incorporate some measure of risk to rule out, for instance, policies that achieve a high expected reward at the cost of risky and error-prone actions. As a result, risk-sensitive optimality criteria for MDPs were put forward.
In the last decade, the notion of risk measures has become very popular. Intuitively a risk measure is a function that maps a cost or reward to a real value, and the aim is to minimise the risk measure. Despite the existing work on risk measures in MDPs, there are still many questions to be answered in this area: what is the computational complexity, how to develop efficient algorithms, or how to provide effective tool support, just to name a few. The proposed research is to address those questions in depth.
The results will shed some lights on whether the risk measure minimisation problem can be performed efficiently at all. If so, how to perform efficiently? If not, is there an efficient algorithm to compute a close enough solution? On top of the algorithmic results, how to develop a ready-to-use tool to calculate the minimal risks? The ultimate goal is to provide a strategy to guide the decision making so that risks are minimised. This can, for instance, help people distribute their asset portfolio, or give advice on the manufacturing processing, or control the robot to deliver a safe and cost-economic path, etc.
|