Complexity of Programs and Algorithms

Abstract

For a given problem, it is essential to compare different programs and algorithms in terms of efficiency and/or accuracy. Efficiency refers to the speed of programs and algorithms, as well as the amount of memory required by them. For both speed and memory usage, the efficiency of a program/algorithm naturally depends on its inputs. In order to compare the efficiency of programs and algorithms, we need to analyze them by estimating or measuring their time and/or memory costs. Estimations are as important as measurements; because, we often need to know the cost of a program before writing and using it. In addition, we would like to know the cost when different computers are used, rather than relying on a measurement on a specific computer. Consequently, it is not surprising that calculating costs of programs and algorithms has become a major area in computer science. In this chapter, we use a systematic approach to understand and estimate costs of programs and algorithms. We start with a rigorous analysis by counting each operation and each piece of storage in programs. When these costs are stated as functions of input sizes (and sometimes input values), the resulting expressions are called time and memory complexities. Then, we simplify these expressions using asymptotic techniques, leading a very important concept, namely, the order. After some practices, we derive shortcuts to calculate the orders of programs and algorithms by finding critical lines and items in the codes.