High Performance Computing and Communication (HPCC)

Abstract

As announced in the hardware Chap. 5, there is a relatively new paradigm of computation in which a single very large application problem is partitioned into subproblems which can be computed concurrently by being distributed over a network of computers, called a cluster, usually a large-scale local area network (LAN) in which nodes of the network can be connected directly through large switches to build a message-passing system. The given application problem is computationally intensive and would take too long to solve on an existing single computer, however fast it may run. For certain problem structures it is possible to partition the problem into components which can be run concurrently to achieve a speed-up that makes the solution feasible. An example is the solution of a linear system Ax = b where A is a nxn matrix and b is an n-dimensional vector. The Jacobi iterative method can be programmed to work on K groups of m components of x concurrently where n = Km. The K groups are computed on K nodes of the cluster. The individual K computers installed as the nodes in the network, usually pc’s, operate independently on their assigned subproblems for the most part, but at various points in their computations they must communicate with each other by a message-passing protocol which permits the cooperative exchange of intermediate results needed in the various subproblems. This paradigm is designated as high performance computing and communication (HPCC).

Appendix

Cloud Computing

• E. K. Blum 

1. Department of Mathematics, University of Southern California, Los Angeles, CA, USA

   E. K. Blum

In the preceding chapter a new computing paradigm, called cluster computing was presented. It involves distributing a computational problem over a computer network. The chapter also referred to a paradigm called cloud computing, which has been promoted recently. Cluster computing under certain conditions of management can be regarded as an example of cloud computing, since both paradigms involve distributing computation over an assemblage of computational resources. However, the term cloud computing is not well-defined if at all. The purpose of this brief appendix is to clarify the current meaning of the term.

A description has been formulated by the National Institute of Standards (NIST) and we are inclined to agree with the NIST description coauthored by Peter Mell and Tim Grance. We give a brief summary of their description, with our own evaluations added. We note that there have been several articles on Cloud Computing published in the ACM Communications, for example in 04/2010, vol.53, No.4, pp.50–58. However, these articles fail to give a precise definition and are, like the following account, only a sketchy outline. This reflects the newness and rapidly changing formations of clouds.

We agree with NIST that cloud computing is an “evolving paradigm” and we further understand that it is at present (and possibly for the immediate future) incompletely defined. At present, the cloud computing industry (those organizations which claim to be engaged in cloud computing) works with many different models and approaches. Indeed some vendors seem to confuse it with cluster computing, which is discussed in our preceding chapter on high performance computing (HPCC). Indeed, cluster computing may be regarded as a case of cloud computing under certain conditions. This indicates that the “cloud” aspect, like clusters, certainly involves a distribution of computing resources. How wide or organized this distribution can be remains unspecified. Most models of cloud computing assume a cloud service provider which provides a shared pool (a “cloud”) of computing resources (servers, storage units, networked pc’s, application software etc.) in a manner that can be rapidly accessed and released by a client with minimal overall management or service provider action. An authorized client of a cloud service can unilaterally request and acquire resources as needed, such as server time, storage space etc. without approval of the request by the cloud service provider. Presumably the quantity of resources available in the cloud permits this. The access to networked components of the cloud is through standard networking techniques that allow mixed client platforms of devices like mobile phones, laptops, pc’s and PDA’s. The cloud service provider can serve multiple clients by providing resources in a dynamic mode according to changing clients demands. Note that this dynamic allocation of resources to clients is also a feature of large clusters. (See the preceding chapter on HPCC.) The quantity of resources is usually so large as to seem virtually unlimited to the client. Resource utilization can be monitored and reported to the service and the client. NIST recognizes three explicit categories of cloud service as follows.

Cloud Software as a Service (SaaS). This service allows a client to use applications software already running on the cloud infrastructure. Various client devices (e.g. pc’s or laptops) can access applications software as web pages through a web browser. Cloud Infrastructure as a Service (IaaS). Infrastructure such as storage networks or computer networks is provided to a client on which the client can run arbitrary software of his own, including operating systems and applications. The client does not control the infrastructure except for possible limited control of some network components such as host firewalls. Cloud Platform as a Service (PaaS). Client has control over deployed applications but cannot manage/control cloud infrastructure. There are four models of cloud deployment: (1) Private: infrastructure dedicated to a singe organization; (2) Community: infrastructure shared by several cooperating organizations; (3) Public: infrastructure available to the general public and owned by a company selling cloud services; (4) Hybrid: a mix of two or more of the first three but using a proprietary technology that permits load balancing between the three. Despite its lack of precise definition cloud computing is an active new mode of computation. For further examples see Chap. 9.