# Optimal Node Visitation in Acyclic Stochastic Digraphs with Multi-threaded Traversals and Internal Visitation Requirements

## Abstract

The original definition of the problem of optimal node visitation (ONV) in acyclic stochastic digraphs concerns the identification of a routing policy that will enable the visitation of each *leaf* node a requested number of times, while minimizing the expected number of the graph traversals. The original work of Bountourelis and Reveliotis (2006) formulated this problem as a Stochastic Shortest Path (SSP) problem, and since the state space of this SSP formulation is exponentially sized with respect to the number of the target nodes, it also proposed a suboptimal policy that is computationally tractable and asymptotically optimal. This paper extends the results of Bountourelis and Reveliotis (2006) to the cases where (i) the tokens traversing the graph can “*split*” during certain transitions to a number of (sub-)tokens, allowing, thus, the satisfaction of many visitation requirements during a single graph traversal, and (ii) there are additional visitation requirements attached to the *internal* graph nodes, which, however, can be served only when the visitation requirements of their successors have been fully met. In addition, the presented set of results establishes stronger convergence properties for the proposed suboptimal policies, and it provides a formal complexity analysis of the considered ONV formulations. From a practical standpoint, the extension of the original results performed in this paper enables their effective usage in the application domains that motivated the ONV problem, in the first place.

## Keywords

Optimal node visitation Acyclic stochastic digraphs Stochastic shortest path problems Stochastic scheduling Fluid relaxation## Notes

### Acknowledgement

This work was partially supported by NSF grants DMI-MES-0318657 and CMMI-0619978.

## References

- Bertsekas DP (1999) Nonlinear programming (2nd ed). Athena Scientific, BelmontMATHGoogle Scholar
- Bertsekas DP (2005) Dynamic programming and optimal control (3rd ed). Athena Scientific, BelmontMATHGoogle Scholar
- Bertsimas D, Gamarnik D (1999) Asymptotically optimal algorithms for job shop scheduling and packet switching. J Algorithms 33:296–318MATHCrossRefMathSciNetGoogle Scholar
- Bertsimas D, Sethuraman J (2002) From fluid relaxations to practical algorithms for job shop scheduling: the makespan objective. Math Program 92:61–102MATHCrossRefMathSciNetGoogle Scholar
- Bertsimas D, Tsitsiklis JN (1997) Introduction to linear optimization. Athena Scientific, BelmontGoogle Scholar
- Billingsley P (1968) Convergence of probability measures. Wiley, New YorkMATHGoogle Scholar
- Bountourelis T, Reveliotis S (2006) Optimal node visitation in acyclic stochastic digraphs. In: Proceedings the 8th intl workshop on discete event systems (WODES’06), IFAC, Ann Arbor, July 2006, pp 358–365Google Scholar
- Bountourelis T, Reveliotis S (2007) Rollout policies for the problem of optimal node visitation in acyclic stochastic digraphs. In: European control conference 2007. IEEE, Piscataway, pp 2456–2463Google Scholar
- Bountourelis T, Reveliotis SA (2008) Customized learning algorithms for episodic tasks with acyclic state spaces. School of Industrial & Systems Eng., Georgia Tech, Tech RepGoogle Scholar
- Chen H, Yao DD (2001) Fundamentals of queueing networks: performance, asymptotics, and optimization. Springer, New YorkMATHGoogle Scholar
- Dai JG (1999) Stability of fluid and stochastic processing networks. Center for Mathematical Physics and Stochastics, University of Aarhus, Denmark, Tech Rep ISSN 1398-7957Google Scholar
- Gut A (1974) On the moments and limit distibutions of some first passage times. Ann Probab 2(2):277–308MATHCrossRefMathSciNetGoogle Scholar
- Meyn S (2008) Control techniques for complex networks. Cambridge University Press, CambridgeMATHGoogle Scholar
- Niño–Mora J (2001) Stochastic scheduling. In: Floudas CA, Pardalos PM (eds) Encyclopedia of optimization. Kluwer, Dordrecht, pp 367–372Google Scholar
- Papadimitriou CH (1985) Games against nature. J Comput Syst Sci 31:288–301MATHCrossRefMathSciNetGoogle Scholar
- Pinedo M (2002) Scheduling: theory, algorithms and systems (2nd ed). Prentice Hall, Upper Saddle RiverMATHGoogle Scholar
- Reveliotis SA (2007) Uncertainty management in optimal disassembly planning through learning-based strategies. IIE Trans 39:645–658CrossRefGoogle Scholar
- Reveliotis SA, Bountourelis T (2007) Efficient PAC learning for episodic tasks with acyclic state spaces. J Discrete Event Syst Theory Appl 17:307–327MATHCrossRefMathSciNetGoogle Scholar
- Reveliotis SA, Bountourelis T (2008) Optimal flow control in acyclic networks with uncontrollable routings and precedence constraints. School of Industrial & Systems Eng., Georgia Tech (under review in IEEE Trans Automat Contr), Tech RepGoogle Scholar
- Ross SM (1996) Stochastic processes. Wiley, New YorkMATHGoogle Scholar