This paper studies optimal switching on and off of the entire service capacity of an M/M/∞ queue with holding, running and switching costs. The running costs depend only on whether the system is on or off, and the holding costs are linear. The goal is to minimize average costs per unit time. The main result is that an average-cost optimal policy either always runs the system or is an (M, N)-policy defined by two thresholds M and N, such that the system is switched on upon an arrival epoch when the system size accumulates to N and is switched off upon a departure epoch when the system size decreases to M. It is shown that this optimization problem can be reduced to a problem with a finite number of states and actions, and an average-cost optimal policy can be computed via linear programming. An example, in which the optimal (M, N)-policy outperforms the best (0, N)-policy, is provided. Thus, unlike the case of single-server queues studied in the literature, (0, N)-policies may not be average-cost optimal.

Consider n servers having different exponential service distributions. All servers are initially busy and there are m customers waiting in queue in an ordered line. A server becoming idle is offered to the first in line, if rejected it is then offered to the second, and so on. The objective of each person in line is to minimize their expected time until service completion. We give a simple approach for finding the optimal policy and also show that this policy also minimizes the expected sum of the times the customers spend in the system.

We consider a two-dimensional reflecting random walk on the non-negative integer quadrant. It is assumed that this reflecting random walk has skip-free transitions. We are concerned with its time-reversed process assuming that the stationary distribution exists. In general, the time-reversed process may not be a reflecting random walk. In this paper, we derive necessary and sufficient conditions for the time-reversed process also to be a reflecting random walk. These conditions are different from but closely related to the product form of the stationary distribution.

Generalized quantiles of a random variable were defined as the minimizers of a general asymmetric loss function, which include quantiles, expectiles and M-quantiles as their special cases. Expectiles have been suggested as potentially better alternatives to both Value-at-Risk and expected shortfall risk measures. In this paper, we first establish the first-order expansions of generalized quantiles for extreme risks as the confidence level α↑ 1, and then investigate the first-order and/or second-order expansions of expectiles of an extreme risk as α↑ 1 according to the underlying distribution belonging to the max-domain of attraction of the Fréchet, Weibull, and Gumbel distributions, respectively. Examples are also presented to examine whether and how much the first-order expansions have been improved by the second-order expansions.

The concept of stochastic precedence between two real-valued random variables has often emerged in different applied frameworks. In this paper, we analyze several aspects of a more general, and completely natural, concept of stochastic precedence that also had appeared in the literature. In particular, we study the relations with the notions of stochastic ordering. Such a study leads us to introducing some special classes of bivariate copulas. Motivations for our study can arise from different fields. In particular, we consider the frame of Target-Based Approach in decisions under risk. This approach has been mainly developed under the assumption of stochastic independence between “Prospects” and “Targets”. Our analysis concerns the case of stochastic dependence.

In this paper, we provide a new technique for analyzing the nonstationary Erlang loss queueing model with abandonment. Our method uniquely combines the use of the functional Kolmogorov forward equations with the well-known Gram-Charlier series expansion from the statistics literature. Using the Gram-Charlier series expansion, we show that we can estimate salient performance measures of the loss queue such as the mean, variance, skewness, kurtosis, and blocking probability. Lastly, we provide numerical examples to illustrate the effectiveness of our approximations.

We study the steady-state queue-length vector in a multi-class queue with relative priorities. Upon service completion, the probability that the next served customer is from class k is controlled by class-dependent weights. Once a customer has started service, it is served without interruption until completion. We establish a state-space collapse for the scaled queue-length vector in the heavy-traffic regime, that is, in the limit the scaled queue-length vector is distributed as the product of an exponentially distributed random variable and a deterministic vector. We observe that the scaled queue length reduces as classes with smaller mean service requirement obtain relatively larger weights. We finally show that the scaled waiting time of a class-k customer is distributed as the product of two exponentially distributed random variables.

We analyze a continuous review inventory model with the marginal carrying cost of a unit of inventory given by an increasing function of its shelf age and the marginal delay cost of a backlogged demand unit by an increasing function of its delay duration. We show that, under a minor restriction, an (r, q)-policy is optimal when the demand process is a renewal process, and a state dependent (r, q)-policy is optimal when the demand is a Markov-modulated renewal process. We also derive various monotonicity properties for the optimal policy parameters r* and r* + q*.

Two stochastic knapsack problem (SKP) models are considered: the static broken knapsack problem (BKP) and the SKP with simple recourse and penalty cost problem. For both models, we assume: the knapsack has a constant capacity; there are n types of items and each type has an infinite supply; a type i item has a deterministic reward vi and a random weight with known distribution Fi. Both models have the same objective to maximize expected total return by finding the optimal combination of items, that is, quantities of items of each type to be put in knapsack. The difference between the two models is: if knapsack is broken when total weights of items put in knapsack exceed the knapsack's capacity, for the static BKP model, all existing rewards would be wiped out, while for the latter model, we could still keep the existing rewards in knapsack but have to pay a fixed penalty plus a variant cost proportional to the overcapacity amount. This paper also discusses the special case when knapsack has an exponentially distributed capacity.

Generally, the multi-armed has been studied under the setting that at each time step over an infinite horizon a controller chooses to activate a single process or bandit out of a finite collection of independent processes (statistical experiments, populations, etc.) for a single period, receiving a reward that is a function of the activated process, and in doing so advancing the chosen process. Classically, rewards are discounted by a constant factor β∈(0, 1) per round.

In this paper, we present a solution to the problem, with potentially non-Markovian, uncountable state space reward processes, under a framework in which, first, the discount factors may be non-uniform and vary over time, and second, the periods of activation of each bandit may be not be fixed or uniform, subject instead to a possibly stochastic duration of activation before a change to a different bandit is allowed. The solution is based on generalized restart-in-state indices, and it utilizes a view of the problem not as “decisions over state space” but rather “decisions over time”.

We consider a queueing loss system with heterogeneous skill based servers with arbitrary distributions. We assume Poisson arrivals, with each arrival having a vector indicating which of the servers are eligible to serve it. Arrivals can only be assigned to a server that is both idle and eligible. We assume arrivals are assigned to the idle eligible server that has been idle the longest and derive, up to a multiplicative constant, the limiting distribution for this system. We show that the limiting probabilities of the ordered list of idle servers depend on the service time distributions only through their means. Moreover, conditional on the ordered list of idle servers, the remaining service times of the busy servers are independent and have their respective equilibrium service distributions. We also provide an algorithm using Gibbs sampler Markov Chain Monte Carlo method for estimating the limiting probabilities and other desired quantities of this system.

One problem of wide interest involves estimating expected crossing-times. Several tools have been developed to solve this problem beginning with the works of Wald and the theory of sequential analysis. Deriving the explicit close form solution for the expected crossing times may be difficult. In this paper, we provide a framework that can be used to estimate expected crossing times of arbitrary stochastic processes. Our key assumption is the knowledge of the average behavior of the supremum of the process. Our results include a universal sharp lower bound on the expected crossing times. Furthermore, for a wide class of time-homogeneous, Markov processes, including Bessel processes, we are able to derive an upper bound E[a(Tr)]≤2r, which implies that sup r>0|((E[a(Tr)]−r)/r)|≤1, where a(t)=E[sup tXt] with {Xt}t≥0 be a non-negative, measurable process. This inequality motivates our claim that a(t) can be viewed as a natural clock for all such processes. The cases of multidimensional processes, non-symmetric and random boundaries are handled as well. We also present applications of these bounds on renewal processes in Example 10 and other stochastic processes.

We analyze optimal replacement and repair problems of semi-Markov missions that are composed of phases with random sequence and durations. The mission process is the minimal semi-Markov process associated with a Markov renewal process. The system is a complex one consisting of non-identical components whose failure properties depend on the mission process. We prove some monotonicity properties for the optimal replacement policy and analyze the optimal repair problem under different cost structures.

This paper defines and studies the down entrance state (DES) and the restart entrance state (RES) classes of quasi-skip free (QSF) processes specified in terms of the nonzero structure of the elements of their transition rate matrix Q. A QSF process is a Markov chain with states that can be specified by tuples of the form (m, i), where $m \in {\open Z}$ represents the “current” level of the state and $i \in {\open Z}^{+}$ the current phase of the state, and its transition probability matrix Q does not permit one-step transitions to states that are two or more levels away from the current state in one direction of the level variable m. A QSF process is a DES process if and only if one step “down” transitions from a level m can only reach a single state in level m − 1, for all m. A QSF process is a RES process if and only if one step “up” transitions from a level m can only reach a single set of states in the highest level M2, largest of all m.

We derive explicit solutions and simple truncation bounds for the steady-state probabilities of both DES and RES processes, when in addition Q insures ergodicity. DES and RES processes have applications in many areas of applied probability comprising computer science, queueing theory, inventory theory, reliability, and the theory of branching processes. To motivate their applicability we present explicit solutions for the well-known open problem of the M/Er/n queue with batch arrivals, an inventory model, and a reliability model.

Much of the work relating to control charts is on monitoring observations for changes in mean. Here, for the first time, we develop a class of cusum-type statistics to monitor any aspect of a process (for example, mean, variance, skewness or kurtosis). This development is based on recently published papers by the authors.

We propose a method for approximating equivalent local volatility functions of stochastic volatility models. Enlightened by the theory of generalized Wiener functionals proposed by Watanabe and Yoshida (1987, 1992), our key technique is to propose a closed-form expansion of conditional expectations involving marginal distributions generated by stochastic differential equations. A numerical test and an illustration of application are provided to demonstrate the efficiency of our approach.

Alternating renewal processes have been widely used to model social and scientific phenomenal where independent “on” and “off” states alternate. In this paper, we study a model where the value of a process cumulates and declines according to two modes of compound Poisson processes with respect to an underlying alternating renewal process. The model discussed in the present paper can be used as a revenue management model applied to inventory or to finance. The exact distribution of the process is derived as well as the double Laplace transform with respect to the level and time of the process.

Recently, Chen, Hsiau & Yang [1] proposed a new two-urn model with red and white balls and showed that the fractions of red balls in both urns converge almost surely to the same limit. We extend the results for the two-urn model to the q-urn model (q≥3) with similar dynamics of drawing and adding balls. We use matrix forms and martingale theory to show that the fractions of red balls in all urns converge almost surely to the same limit.

We show that overshoots over Erlang random variables give rise to a natural generalization of the stationary excess operator and its iterates. The new operators can be used to derive expansions for the expectation Eg(X) of a non-negative random variable, similar to Taylor-like expansions encountered when studying stationary excess operators.

In this paper, we consider a financial market with an asset exposed to a risk inducing a jump in the asset price, and which can still be traded after the default time. We use a default-intensity modeling approach, and address in this incomplete market context the problem of maximization of expected utility from terminal wealth for logarithmic, power and exponential utility functions. We study this problem as a stochastic control problem both under full and partial information. Our contribution consists of showing that the optimal strategy can be obtained by a direct approach for the logarithmic utility function, and the value function for the power (resp. exponential) utility function can be determined as the minimal (resp. maximal) solution of a backward stochastic differential equation. For the partial information case, we show how the problem can be divided into two problems: a filtering problem and an optimization problem. We also study the indifference pricing approach to evaluate the price of a contingent claim in an incomplete market and the information price for an agent with insider information.