State-dependent Control of a Single Stage Hybrid System with Poisson Arrivals



We consider a single-stage hybrid manufacturing system where jobs arrive according to a Poisson process. These jobs undergo a deterministic process which is controllable. We define a stochastic hybrid optimal control problem and decompose it hierarchically to a lower-level and a higher-level problem. The lower-level problem is a deterministic optimal control problem solved by means of calculus of variations. We concentrate on the stochastic discrete-event control problem at the higher level, where the objective is to determine the service times of jobs. Employing a cost structure composed of process costs that are decreasing and strictly convex in service times, and system-time costs that are linear in system times, we show that receding horizon controllers are state-dependent controllers, where state is defined as the system size. In order to improve upon receding horizon controllers, we search for better state-dependent control policies and present two methods to obtain them. These stochastic-approximation-type methods utilize gradient estimators based on Infinitesimal Perturbation Analysis or Imbedded Markov Chain techniques. A numerical example demonstrates the performance improvements due to the proposed methods.


Poisson arrivals Hierarchical decomposition Receding horizon control State-dependent control policy Infinitesimal perturbation analysis Imbedded Markov chains Stochastic approximation 


  1. Cassandras CG, Lafortune S (2007) Introduction to discrete event systems. SpringerGoogle Scholar
  2. Cassandras CG, Lygeros J (2006) Stochastic hybrid systems (automation and control engineering). CRC PressGoogle Scholar
  3. Gokbayrak K, Cassandras CG (2000a) A hierarchical decomposition method for optimal control of hybrid systems. In: Proceedings of 39th IEEE conference on decision and control, pp 1816–1821Google Scholar
  4. Gokbayrak K, Cassandras CG (2000b) Hybrid controllers for hierarchically decomposed systems. In: Proc. of 2000 hybrid system control conference, pp 117–129Google Scholar
  5. Bryson AE, Ho YC (1975) Applied optimal control. Hemisphere Publishing CoGoogle Scholar
  6. Cassandras CG, Mookherjee R (2003a) Receding horizon control for a class of hybrid systems with event uncertainties. In: Proc. of 2003 American control conf, pp 5197–5202Google Scholar
  7. Cassandras CG, Mookherjee R (2003b) Properties of receding horizon controllers for some hybrid systems with event uncertainties. In: Proc. 2003 IFAC conference on analysis and design of hybrid systems, pp 413–418Google Scholar
  8. Cassandras CG, Mookherjee R (2003c) Receding horizon optimal control for some stochastic hybrid systems. In: Proc. of 42nd IEEE conference decision and control, pp 2162–2167Google Scholar
  9. Kushner HJ, Yin GG (2003) Stochastic approximation and recursive algorithms and applications. Springer-Verlag, New YorkMATHGoogle Scholar
  10. Glasserman P (1991) Gradient estimation via perturbation analysis. Kluwer Academic PublishersGoogle Scholar
  11. Ho Y, Cao X (1991) Perturbation analysis of discrete event dynamic systems. Kluwer Academic PublishersGoogle Scholar
  12. Suri R, Zazanis MA (1988) Perturbation analysis gives strongly consistent sensitivity estimates for the M/G/1 queue. Manag Sci 34(1):39–64MathSciNetMATHCrossRefGoogle Scholar
  13. L’Ecuyer P, Glynn PW (1994) Stochastic optimization by simulation: Convergence proofs for the GI/G/1 queue in steady-state. Manag Sci 40(11):1562–1578MATHCrossRefGoogle Scholar
  14. Chong EKP, Ramadge PJ (1992) Convergence of recursive optimization algorithms using ipa derivative estimates. Discrete Event Dyn Syst, Theory and Applications 1:339–372MATHGoogle Scholar
  15. Fu MC (1990) Convergence of the stochastic approximation algorithm for the gi/g/1 queue using infinitesimal perturbation analysis. J Optim Theory Appl 65:149–160MathSciNetMATHCrossRefGoogle Scholar
  16. Zazanis MA, Suri R (1994) Perturbation analysis of the GI/GI/1 queue. Queueing Syst 18(3):199–248MathSciNetMATHCrossRefGoogle Scholar
  17. Kendall DG (1951) Some problems in the theory of queues. J R Stat Soc B (Methodological) 13(2):151–185MathSciNetMATHGoogle Scholar
  18. Wolff RW (1982) Poisson arrivals see time averages. Oper Res 30(2):223–231MathSciNetMATHCrossRefGoogle Scholar
  19. Foster FG (1953) On the stochastic matrices associated with certain queuing processes. Ann Math Stat 24(3):355–360MathSciNetMATHCrossRefGoogle Scholar
  20. Harris CM (1967) Queues with state-dependent stochastic service rates. Oper Res 15(1):117–130MATHCrossRefGoogle Scholar
  21. Heyman DP, Sobel MJ (2003) Stochastic models in operations research, vol 1. Dover PublicationsGoogle Scholar
  22. Pepyne DL, Cassandras CG (1998) Modeling, analysis, and optimal control of a class of hybrid systems. Discrete Event Dyn Syst, Theory and Applications 8(2):175–201MathSciNetMATHCrossRefGoogle Scholar
  23. Pepyne DL, Cassandras CG (2000) Optimal control of hybrid systems in manufacturing. Proc. IEEE 88(7):1108–1123CrossRefGoogle Scholar
  24. Cassandras CG, Pepyne DL, Wardi Y (2001) Optimal control of a class of hybrid systems. IEEE Trans Automat Contr 46(3):398–415MathSciNetMATHCrossRefGoogle Scholar
  25. Wardi Y, Cassandras CG, Pepyne DL (2001) A backward algorithm for computing optimal controls for single-stage hybrid manufacturing systems. Int J Prod Res 39(2):369–393MATHCrossRefGoogle Scholar
  26. Cho YC, Cassandras CG, Pepyne DL (2001) Forward decomposition algorithms for optimal control of a class of hybrid systems. Int J Robust Nonlinear Control 11:497–513MathSciNetMATHCrossRefGoogle Scholar
  27. Zhang P, Cassandras CG (2002) An improved forward algorithm for optimal control of a class of hybrid systems. IEEE Trans Automat Contr 47(10):1735–1739MathSciNetCrossRefGoogle Scholar
  28. Robbins H, Monro S (1951) A stochastic approximation method. Ann. Math. Stat 22:400–407MathSciNetMATHCrossRefGoogle Scholar
  29. Gross D, Shortle JF, Thompson JM, Harris CM (2008) Fundamentals of queueing theory, 4th edn. WileyGoogle Scholar
  30. Little JDC (1961) A proof for the queueing formula l=λw. Oper Res 9(3):383–387MathSciNetMATHCrossRefGoogle Scholar
  31. Trigg DW, Leach AG (1967) Exponential smoothing with an adaptive response rate. Oper Res Q 18(1):53–59CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Department of Industrial EngineeringBilkent UniversityAnkaraTurkey

Personalised recommendations