Présidents de session
Sequential decision-making under uncertainty: Contributed talks
- Michel De Lara (Ecole des Ponts ParisTech)
Sequential decision-making under uncertainty: Contributed talks
- Jingui Xie (Technical University of Munich)
Sequential decision-making under uncertainty: Contributed talks
- Paul Malisani (IFP Energies nouvelles)
Sequential decision-making under uncertainty: Contributed talks
- David Wozabal (Vrije Universiteit Amsterdam)
Sequential decision-making under uncertainty: Contributed talks
- Hoda Bidkhori (George Mason University)
Sequential decision-making under uncertainty: New developments in Stochastic Dual Dynamic Programming
- Bernardo Freitas Paulo da Costa (Fundação Getulio Vargas)
Lot-Sizing is a class of combinatorial optimization problems encountered in industrial production planning. This work addresses the stochastic capacitated lot-sizing problem with inventory bounds and lost sales. A production problem with uncertain demand is investigated with the assumption that decisions can be regularly updated to adjust to the actual demand being progressively revealed. We...
Energy planning plays a fundamental role in managing the generation resources in a power system. This planning must meet both present and future energy demands, considering various operational, electrical, environmental, political, and other constraints. The main objective is to allocate resources in a way that minimizes costs while mitigating risks associated with future uncertainties,...
In decision problems under incomplete information, actions (identified to payoff vectors indexed by states of nature) and beliefs are naturally paired by bilinear duality. We exploit this duality to analyze the interpersonal comparison of the value of information, using concepts and tools from convex analysis. We characterize each decision-maker (DM) by a closed convex lower set, the...
In this paper, we study the stochastic casualty response planning problem and propose a multi-stage stochastic programming model, where initial decisions—such as the location of alternative care facilities (ACFs) and rescue vehicle assignments—are fixed, while patient assignments and allocations of apheresis machines (AM) for blood extraction are updated dynamically as uncertainty unfolds. In...
Multiarmed bandit problems (MABs) present a class of optimal control problems well-suited for modeling resource allocation under uncertainty. This study explores the application of MABs in the context of clinical trial design. While traditional risk-neutral MABs aim to maximize the expected total number of effective treatments, this study considers the limitations of this objective, as...
Health organizations (society) prefer to recommend universal screening guidelines to at-risk individuals (patients) for various diseases, with coverage typically provided by third-party payers. However, patients are heterogeneous both in disease risk and disutility associated with screening, resulting in varying health outcomes. Infrequent recommendations leave at-risk patients to decide if to...
Deciding when to stop medical treatment with uncertain outcomes and predictions is a critical challenge in intensive care units. This research develops a risk-sensitive approach to optimal medical stopping decisions by integrating outcome variability into the decision-making process and incorporating predictive information about the next state. We model the problem using a risk-sensitive...
An influence diagram is a graph representation of a decision problem that models interdependencies between random events, consequences, and decisions. Recently, two frameworks have been developed to find the optimal decision strategy by transforming the influence diagram into a mixed-integer linear program (MILP). Decision programming (Salo et al., EJOR 299/2, 2022) directly translates the...
This paper deals with robust stochastic optimal control problems. The main contribution is an extension of the Progressive Hedging Algorithm (PHA) that enhances out-of-sample robustness while preserving numerical complexity. This extension involves adopting the widespread practice in machine learning of variance penalization for stochastic optimal control problems. Using the Douglas-Rachford...
The Stochastic Dual Dynamic Programming (SDDP) algorithm is widely used to solve multi-stage stochastic problems, such as hydrothermal dispatch in power systems. Due to its iterative nature and the need to handle large volumes of data and multiple future scenarios, SDDP is a computationally intensive method. With the increasing complexity of modern systems and the need to respond to energy...
This study explores multi-armed bandit problems under the premise that the decision-maker possesses prior knowledge of the arms' distributions and knows the finite time horizon. These conditions render the problems suitable for stochastic multistage optimization decomposition techniques. On the one hand, multi-armed bandit algorithms are integral to reinforcement learning and are...
Multistage stochastic linear programming (MSLP) offers a powerful framework for decision-making under uncertainty over time. Sampling-based algorithms provide a practical approach to solving the MSLP problems, particularly in large-scale settings. In this arena, stochastic dual dynamic programming (SDDP) has proven to be very effective. SDDP utilizes randomization to solve a deterministic...
This paper addresses the computation of tight optimistic bounds for multi-stage stochastic optimization problems using information relaxation duality. We introduce a specific class of penalty functions—bi-linear in decisions and the innovations of the underlying stochastic process—to penalize anticipative policies. Our approach provides a generic framework for deriving such bounds, notably...
We study reinforcement learning by combining recent advances in regularized linear programming formulations with the classical theory of stochastic approximation.
Motivated by the challenge of designing algorithms that leverage off-policy data while maintaining on-policy exploration, we propose PGDA-RL, a novel primal-dual Projected Gradient Descent-Ascent algorithm for solving regularized...
An implied distribution of the underlying asset price for the options expiration moment can be obtained from the market option prices [1]. However, exchange-traded options rarely expire more often than once a month. It is not enough for planning dynamic decisions in many cases. In [2] implied calibration of the dynamic ARMA(1,1)-GARCH(1,1) model using market prices of options of different...
Cancer is the second leading cause of death in the world. Unfortunately, the projections from the International Agency for Research on Cancer (IARC) indicate a rising trend for new cancer cases in the following years. Among the various cancer treatments, chemotherapy is one of the most effective treatments for numerous cancer types. It generally contains one or more prescribed molecules...
The increasing penetration of renewable energy sources in power systems amplifies the need for storage to manage their inherent intermittency. In this context, evaluating the opportunity cost of stored energy—commonly referred to as usage values—becomes essential. These values can be computed by solving a multistage stochastic optimization problem, where uncertainty arises from net demand (the...
Exactly and asymptotically optimal algorithms are developed for robust detection of changes in non-stationary processes. In non-stationary processes, the distribution of the data after change varies with time. The decision maker does not have access to precise information on the post-change distribution. It is shown that if the post-change non-stationary family has a distribution that is least...
We consider the generation of cuts in stochastic dual dynamic programming (SDDP) for multistage stochastic linear programming problems with stagewise dependent uncertainty in the right-hand side described by log-linear (or geometric) autoregressive processes. We show that it is possible to develop tractable closed-form cut formulas in this case. The cuts are linear in all decision variables,...
Hydropower scheduling is an important application of stochastic dynamic programming, involving large optimization problems with intertemporal dependency and complex dynamics. The nature of this problem is naturally seasonal (and therefore cyclic), and even though most real-world use cases usually apply discretization strategies (such as monthly, weekly or daily time steps) it is reasonable to...
We develop a new stopping rule for the stochastic dual dynamic programming (SDDP) algorithm based on sequential testing. Traditional stopping rules are based on statistical tests that assess whether the optimality gap in a given iteration is smaller than a pre-defined precision. However, repeated use of such single-iteration tests invalidates the overall validity of the stopping rule, because...
This work addresses the challenges of applying Stochastic Dual Dynamic Programming (SDDP) to infinite-horizon hydroelectric water management problems with continuous state and control spaces. While SDDP has proven effective in finite-horizon settings, its extension to the infinite-horizon case with a discount factor close to one introduces numerical difficulties when the discount rate is close...
Tri-level defender-attacker game models are a well-studied method for
determining how best to protect a system (e.g., a transportation network) from attacks.
Existing models assume that defender and attacker actions have a perfect effect, i.e.,
system components hardened by a defender cannot be destroyed by the attacker, and
attacked components always fail. Because of these assumptions,...
