Encyclopedia of Database Systems

Living Edition
| Editors: Ling Liu, M. Tamer Özsu

Adaptive Middleware for Message Queuing Systems

  • Christophe TatonEmail author
  • Noel De Palma
  • Sara Bouchenak
Living reference work entry
DOI: https://doi.org/10.1007/978-1-4899-7993-3_1540-2



Distributed database systems are usually built on top of middleware solutions, such as message queuing systems. Adaptive message queuing systems are able to improve the performance of such a middleware through load balancing and queue provisioning.

Historical Background

The use of message oriented middlewares (MOMs) in the context of the Internet has evidenced a need for highly scalable and highly available MOM. A very promising approach to the above issue is to implement performance management as an autonomic software. The main advantages of this approach are: (i) Providing a high-level support for deploying and configuring applications reduces errors and administrator’s efforts. (ii) Autonomic management allows the required reconfigurations to be performed without human intervention, thus improving the system reactivity and saving administrator’s time. (iii) Autonomic management is a means to save hardware resources, as resources can be allocated only when required (dynamically upon failure or load peak) instead of pre-allocated.

Several parameters may impact the performance of MOMs. Self-optimization makes use of these parameters to improve the performance of the MOM. The proposed self-optimization approach is based on a queue clustering solution: a clustered queue is a set of queues each running on different servers and sharing clients. Self-optimization takes place in two parts: (i) the optimization of the clustered queue load-balancing and (ii) the dynamic provisioning of a queue in the clustered queue. The first part allows the overall improvement of the clustered queue performance while the second part optimizes the resource usage inside the clustered queue. Thus the idea is to create an autonomic system that fairly distributes client connections among the queues belonging to the clustered queue and dynamically adds and removes queues in the clustered queue depending on the load. This would allow to use the adequate number of queues at any time.


Clustered Queues

A queue is a staging area that contains messages which have been sent by message producers and are waiting to be read by message consumers. A message is removed from the queue once it has been read. For scalability purpose, a queue can be replicated forming a clustered queue. The clustered queue feature provides a load balancing mechanism. A clustered queue is a cluster of queues (a given number of queue destinations knowing each other) that are able to exchange messages depending on their load. Each queue of a cluster periodically reevaluates its load factor and sends the result to the other queues of the cluster. When a queue hosts more messages than it is authorized to do, and according to the load factors of the cluster, it distributes the extra messages to the other queues. When a queue is requested to deliver messages but is empty, it requests messages from the other queues of the cluster. This mechanism guarantees that no queue is hyper-active while some others are lazy, and tends to distribute the work load among the servers involved in the cluster.

Clustered Queue Performance

Clustered queues are standard queues that share a common pool of message producers and consumers, and that can exchange message to balance the load. All the queues of a clustered queue are supposed to be directly connected to each other. This allows message exchanges between the queues of a cluster in order to empty flooded queues and to fill draining queues.

The clustered queue Q c is connected to N c message producers and to M c message consumers. Q c is composed of standard queues \( {Q}_i\left(i\in \left[1..k\right]\right) \). Each queue Q i is in charge of a subset of N i message producers and of a subset of M i message consumers:
$$ \left\{\begin{array}{l}{N}_c={\displaystyle {\sum}_i{N}_i}\\ {}{M}_c={\displaystyle {\sum}_i{M}_i}\end{array}\right. $$
The distribution of the clients between the queues Q i is described as follows: x i (resp. y i ) is the fraction of message producers (resp. consumers) that are directed to Q i .
$$ \left\{\begin{array}{l}{N}_i={x}_i\mathit{\cdotp}{N}_c\\ {}{M}_i={y}_i\cdotp {M}_c\end{array}\right., \left\{\begin{array}{l}{\displaystyle {\sum}_i{x}_i=1}\\ {}{\displaystyle {\sum}_i{y}_i=1}\end{array}\right. $$
The standard queue Q i to which a consumer or producer is directed to cannot be changed after the client connection to the clustered queue. This way, the only action that may affect the client distribution among the queues is the selection of an adequate queue when the client connection is opened.
The clustered queue Q c is characterized by its aggregate message production rate p c and its aggregate message consumption rate c c . The clustered queue Q c also has a virtual clustered queue length l c that aggregates the length of all contained standard queues:
$$ {l}_c={\displaystyle \sum_i{l}_i}={p}_c-{c}_c, \left\{\begin{array}{l}{p}_c=\sum i {p}_i\\ {}{c}_c=\sum i {c}_i\end{array}\right. $$
The clustered queue length l c obeys to the same law as a standard queue:
  1. 1.

    Q c is globally stable when Δl c  = 0. This configuration ensures that the clustered queue is globally stable. However Q c may observe local unstabilities if one of its queues is draining or is flooded.

  2. 2.

    If Δl c  > 0, the clustered queue will grow and eventually saturate; then message producers will have to wait.

  3. 3.

    If Δl c  < 0, the clustered queue will shrink until it is empty; then message consumers will also have to wait.


Now, considering that the clustered queue is globally stable, several scenarios that illustrate the impact of client distribution on performance are given below.

Optimal client distribution of the clustered queue Q c is achieved when clients are fairly distributed among the k queues Q i . Assuming that all queues and hosts have equivalent processing capabilities and that all producers (resp. consumers) have equivalent message production (resp. consumption) rates (and that all produced messages are equivalent: message cost is uniformly distributed), this means that:
$$ \left\{\begin{array}{l}{x}_i=1/k\\ {}{y}_i=1/k\end{array}\right., \left\{\begin{array}{l}{N}_i=\frac{N_c}{k},\\ {}{M}_i=\frac{M_c}{k}\end{array}\right. $$
In these conditions, all queues Q i are stable and the queue cluster is balanced. As a consequence, there are no internal queue-to-queue message exchanges, and performance is optimal. Queue clustering then provides a quasi-linear speedup.
The worst clients distribution appears when one queue only has message producers or only has message consumers. In the example depicted in Fig. 1, this is realized when:
Fig. 1

Clustered queue Q c

$$ \left\{\begin{array}{l}{x}_1=1\\ {}{y}_1=0\end{array}\right., \left\{\begin{array}{l}{x}_2=0\\ {}{y}_2=1\end{array}\right., \left\{\begin{array}{l}{N}_1={N}_c\\ {}{M}_1=0\end{array}\right., \left\{\begin{array}{l}{N}_2=0\\ {}{M}_2={M}_c\end{array}\right. $$
Indeed, this configuration implies that the whole message production is directed to queue Q 1. Q 1 then forwards all messages to Q 2 that in turn delivers messages to the message consumers.
Local instability is observed when some queues Q i of Q c are unbalanced. This is characterized by a mismatch between the fraction of producers and the fraction of consumers directed to Q i :
$$ {x}_i\ne {y}_i $$
In the example showed in Fig. 1, Q c is composed of two standard queues Q 1 and Q 2. A scenario of local instability can be envisioned with the following clients distribution:
$$ \left\{\begin{array}{l}{x}_1=2/3\\ {}{y}_1=1/3\end{array}\right., \left\{\begin{array}{l}{x}_2=1/3\\ {}{y}_2=2/3\end{array}\right. $$
This distribution implies that Q 1 is flooding and will have to enqueue messages, while Q 2 is draining and will see its consumer clients wait. However the queue cluster Q c ensures the global stability of the system thanks to internal message exchanges from Q 1 to Q 2.

A stable and unfair distribution can be observed when the clustered queue is globally and locally stable, but the load is unfairly balanced within the queues. This happens when the client distribution is non-uniform.

In the example presented in Fig. 1, this can be realized by directing more clients to Q 1 than Q 2:
$$ \left\{\begin{array}{l}{x}_1=2/3\\ {}{y}_1=2/3\end{array}\right., \left\{\begin{array}{l}{x}_2=1/3\\ {}{y}_2=1/3\end{array}\right. $$
In this scenario, queue Q 1 processes two third of the load, while queue Q 2 only processes one third. Suc situation can lead to bad performance since Q 1 may saturates while Q 2 is lazy.

It is worthwhile to indicate that these scenarios may all happen since clients join and leave the system in an uncontrolled way. Indeed, the global stability of a (clustered) queue is under responsability of the application developper. For instance, the queue can be flooded for a period; it is assumed that it will get inverted and draining after, thus providing global stability over time.


The previous scenario of stable and non-optimal distribution raises the question of the capacity of a queue. The capacity C i of standard queue Q i is expressed as an optimal number of clients. The queue load L i is then expressed as the ratio between its current number of clients and its capacity:
$$ {L}_i=\frac{N_i+{M}_i}{C_i} $$
  1. 1.

    L i  < 1: queue Q i is underloaded and thus lazy; the message throughput delivered by the queue can be improved and resources are wasted.

  2. 2.

    L i  > 1: queue Q i is overloaded and may saturate; this induces a decreased message throughput and eventually leads to thrashing.

  3. 3.

    L i  = 1: queue Q i is fairly loaded and delivers its optimal message throughput.

These parameters and indicators are transposed to queue clusters. The clustered queue Q c is characterized by its aggregated capacity C c and its global load L c :
$$ {C}_c={\displaystyle \sum_i{C}_i}, {L}_c=\frac{N_c+{M}_c}{C_c}=\frac{{\displaystyle {\sum}_i{L}_i\cdotp {C}_i}}{{\displaystyle {\sum}_i{C}_i}} $$
The load of a clustered queue obeys to the same law as the load of a standard queue.
However a clustered queue allows to control k, the number of inside standard queues, and thus to control its aggregated capacity \( {C}_c={\displaystyle {\sum}_{i=1}^k{C}_i} \). This control is indeed operated with a re-evaluation of the clustered queue provisioning.
  1. 1.

    When L c  < 1, the clustered queue is underloaded: if the clients distribution is optimal, then all the standard queues inside the cluster will be underloaded. However, as the client distribution may be non-optimal, some of the single queues may be overloaded, even if the cluster is globally lazy. If the load is too low, then some queues may be removed from the cluster.

  2. 2.

    When L c  > 1, the clustered queue is overloaded: even if the distribution of clients over the queues is optimal, there will exist at least one standard queue that will be overloaded. One way to handle this case is to re-provision the clustered queue by inserting one or more queues into the cluster.


Control Rules for a Self-Optimizing Clustered Queue

The global clients distribution D of the clustered queue Q c is captured by the fractions of message producers x i and consumers y i . The optimal clients distribution D opt is realized when all queues are stable \( \left(\forall i\ {x}_i={y}_i\right) \) and when the load is fairly balanced over all queues \( \left(\forall i,j{x}_i={x}_j,{y}_i={y}_j\right) \). This implies that the optimal distribution is reached when \( {x}_i={y}_i=1/k \).
$$ D = \left[\begin{array}{cc}\hfill {x}_1\hfill & \hfill {y}_1\hfill \\ {}\hfill \vdots \vdots \hfill & \hfill \hfill \\ {}\hfill {x}_k\hfill & \hfill {y}_k\hfill \end{array}\right], {D}_{opt}=\left[\begin{array}{cc}\hfill 1/k\hfill & \hfill 1/k\hfill \\ {}\hfill \vdots \vdots \hfill & \hfill \hfill \\ {}\hfill 1/k\hfill & \hfill 1/k\hfill \end{array}\right] $$
Local instabilities are characterized by a mismatch between the fraction of message producers x i and consumers y i on a standard queue. The purpose of this rule is the stability of all standard queues so as to minimize internal queue-to-queue message transfer.
  1. 1.

    [(R 1)] x i  > y i : Q i is flooding with more message production than consumption and should then seek more consumers and/or fewer producers.

  2. 2.

    [(R 2)] x i  < y i : Q i is draining with more message consumption than production and should then seek more producers and/or fewer consumers.

Load balancing rules control the load applied to a single standard queue. The goal is then to enforce a fair load balancing over all queues.
  1. 1.

    [(R 3)] L i  > 1: Q i is overloaded and should avoid accepting new clients as it may degrade its performance.

  2. 2.

    [(R 4)] L i  < 1: Q i is underloaded and should request more clients so as to optimize resource usage.

Global provisioning rules control the load applied to the whole clustered queue. These rules target the optimal size of the clustered queue while the load applied to the system evolves.
  1. 1.

    [(R 5)] L c  > 1: the queue cluster is overloaded and requires an increased capacity to handle all its clients in an optimal way.

  2. 2.

    [(R 6)] L c  < 1: the queue cluster is underloaded and could accept a decrease in capacity.


Key Applications

Adaptive middleware for message queuing systems helps building autonomous distributed systems to improve their performance while minimizing their resource usage, such as distributed Internet services and distributed information systems.


Recommended Reading

  1. 1.
    Aron M, Druschel P, Zwaenepoel W. Cluster reserves: a mechanism for resource management in cluster-based network servers. In: Proceedings of the 2000 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems; 2000. p. 90–101.Google Scholar
  2. 2.
    Menth M, Henjes R. Analysis of the message waiting time for the fioranoMQ JMS server. In: Proceedings of the 23rd International Conference on Distributed Computing Systems; 2006. p. 1.Google Scholar
  3. 3.
    Shen K, Tang H, Yang T, Chu L. Integrated resource management for cluster-based internet services. In: Proceedings of the 5th USENIX Symposium on Operating System Design and Implementation; 2002.Google Scholar
  4. 4.
    Urgaonkar B, Shenoy P. Sharc: managing CPU and network bandwidth in shared clusters. IEEE Trans Parall Distrib Syst. 2004;15(1):2–17.CrossRefGoogle Scholar
  5. 5.
    Zhu H, Ti H, Yang Y. Demand-driven service differentiation in cluster-based network servers. In: Proceedings of the 20th Annual Joint Conference of the IEEE Computer and Communications Societies, vol. 2; 2001. p. 679–88.Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Christophe Taton
    • 1
    Email author
  • Noel De Palma
    • 1
  • Sara Bouchenak
    • 2
  1. 1.INPG – INRIAGrenobleFrance
  2. 2.University of Grenoble I – INRIAGrenobleFrance

Section editors and affiliations

  • Cristiana Amza
    • 1
  1. 1.Dept. of Elec. and Comp. Eng.Univ. of TorontoTorontoCanada