1 Introduction

Next-generation cellular communication systems, or 5G, will be assisted by technologies that produce significant improvements in cell throughput. In recent years, various studies have focused on massive multiple input multiple output (MIMO) systems, which are considered to play a significant role in 5G. Massive MIMO systems are MIMO systems wherein the precoders and/or detectors contain numerous antennas. Such a larger number of antennas enable higher spectral efficiency and energy efficiency to be achieved. Several types of antennas can be used for this purpose, one of which is called a smart antenna. Smart antennas are organizations of numerous antenna elements at base stations (BSs) and mobile stations of wireless communication links, in which signals are appropriately managed, with the purpose of improving the wireless mobile link and increasing the performance of the system.

Such an antenna is a digital antenna used in wireless communication systems and provides the benefit of increased diversity for the BS and/or user equipment. The antenna enables increase of capacity in wireless communication systems by successfully reducing multipath fading and channel interference, which can be realised by concentrating signal radiation only in the anticipated direction and modifying such radiation according to the signal surroundings or varying traffic situations using beamforming techniques.

In wireless communication systems, transmit and receive beamforming is used for signal transmission from BSs with multiple antennas to one or multiple pieces of user equipment that should be covered. The objective of transmit beamforming is to maximise each user’s received signal power while minimising the interference signal power from the other users, hence increasing capacity. This can be achieved by transmitting the same signal from all transmitters with different amplitudes and phases. These multiple versions of the transmitted signal will pass through different MIMO channels such that they are added constructively at the desired users and destructively at other users.

Several other review papers, such as Vouyioukas (2013) and Murray and Zaghloul (2014), have focused on beamforming techniques for MIMO. Vouyioukas (2013) investigated beamforming techniques in MIMO relay networks and procedures that were recently developed for interference mitigation under various network performance challenges, such as complexity and power consumption reduction and capacity improvements. Murray and Zaghloul (2014) reviewed various cognitive beamforming techniques that can be used in MIMO systems. Several algorithms were proposed based on constraints or idealizations of channel state information (CSI) and quality-of-service metrics. The authors evaluated the cognitive beamforming techniques using distributed, joint, and cooperative beamforming strategies based on game theory, genetic algorithms, and neural networks.

Kutty and Sen (2016) concentrated on the use of beamforming techniques for millimetre wave (mm-wave) communications. They provided a significant survey on the evolution and advancements in antenna beamforming for mm-wave communications in the setting of the different requirements for indoor and outdoor communication scenarios, and introduced beamforming techniques generally by announcing some basic concepts of beamforming, including typical beamforming architectures and approaches.

Heath et al. (2016) provided an overview of signal processing for mm-wave wireless communication systems and described the main mm-wave-MIMO architectures including analogue and hybrid beamforming for different types of propagation models. Furthermore, channel estimation algorithms and beam training protocols were reviewed in detail for mm-wave communications. Although the aforementioned surveys and many other surveys have investigated the importance of beamforming for MIMO systems in detail, they did not discuss which types of beamforming techniques can be deployed for massive MIMO systems according to 5G requirements. Thus, this paper is focused on beamforming technique classifications for wireless communication systems and investigation of their effects on massive MIMO systems to determine which optimal categories can be adopted with massive MIMO system requirements.

This paper provides an in-depth overview of up-to-date research on classifications of beamforming techniques that can be deployed for massive MIMO systems. Several key elements are discussed to show the importance of beamforming techniques in reducing and resolving many technical complications that disallow massive MIMO implementation.

In Section 2, a background of massive MIMO systems and the benefits of applying beamforming techniques for massive MIMO systems are presented. In addition, various types of transmitters (precoders) and receivers (detectors) that can be implemented in massive MIMO systems are introduced. Section 3 provides comprehensive details about beamforming approaches and their classifications in physical terms. Switched and adaptive beamforming algorithms and their possessions are discussed in detail to establish which types of techniques are more affordable for massive MIMO systems. In addition, adaptive beamforming optimization algorithms based on azimuth and elevation angles, such as linearly constrained minimum variance (LCMV) and minimum variance distortionless response (MVDR), are presented. Furthermore, hybrid digital/analogue beamforming clarification that can be implemented in massive MIMO systems is discussed in detail. Section 4 is focused on mm-wave bands and their advantages for beamforming techniques in massive MIMO systems as a type of broadband beamforming. Unresolved issues and trends for the future are addressed in Section 5. Finally, conclusions are provided in Section 6.

2 Background

2.1 Massive MIMO systems

MIMO systems have received significant attention owing to the growing number of served users and the increasing demand for large amounts of data. Multi-user MIMO systems might provide a breakthrough technique for improving spectral efficiency in wireless communications. MIMO has become a key technology for future communication systems as the number of requests for wireless services continues to increase, with the spectrum being finite. Recently, numerous in-depth studies have been conducted in the field of multi-user MIMO communication, in which relevant systems are referred to as massive MIMO or large-scale MIMO systems.

Massive MIMO systems are defined as an arrangement of MU-MIMO systems wherein large quantities of antenna elements at BSs and large quantities of antennas at terminals are deployed. In massive MIMO systems, large quantities of antennas (hundreds or thousands) connected to a BS simultaneously work for considerably fewer (tens or hundreds) terminals using similar time and carrier frequency resources (Larsson et al., 2014). Massive MIMO systems can improve the capacity of wireless communication systems 10-fold or more owing to their characteristics and the energy efficiency by approximately 100-fold. The capacity increase enabled by massive MIMO systems is due to the large number of antennas that are implemented. However, using a large number of antennas causes interference problems, which can be mitigated by deploying beamforming antennas instead of conventional antennas.

The definition of beamforming in massive MIMO systems differs slightly from the aforementioned definitions. Beamforming is a signal processing procedure used with multiple arrays of antennas at the transmitter side and/or receiver side to simultaneously send or detect multiple signals from multiple desired terminals to increase system capacity and performance. Beamforming can be realised by assembling the elements in an organised array, in which beams steered toward a specific direction are added and the other beams neglected. Although this technique is not new, it remains reinforced by developed wireless communication system organizations, namely, long-term evolution (LTE) and LTE advanced operators. These operators focus on integrating beamforming techniques into wireless communication systems. The energy efficiency of massive MIMO systems could be increased dramatically by deploying a large quantity of beamforming antenna elements at the BS (Rusek et al., 2013; Larsson et al., 2014; Lu et al., 2014). Recent studies related to 5G wireless systems have largely focused on massive MIMO systems and beamforming solutions. However, beamforming techniques can still contribute to further enhancements of future wireless communication systems.

2.2 Benefits of beamforming in massive MIMO systems

Beamforming is a process formulated to produce the radiated beam patterns of the antennas by completely building up the processed signals in the direction of the desired terminals and cancelling beams of interfering signals. This can be accomplished using a finite impulse response (FIR) filter. FIR filters are beneficial in that their weights can be varied adaptively and applied to obtain the optimum beamforming. The application of beamforming in massive MIMO systems has the following advantages: enhanced energy efficiency, improved spectral efficiency, increased system security, and applicability for mm-wave bands.

2.2.1 Enhanced energy efficiency

The lower power requirements of beamforming antennas for transmitting signals to the intended user and cost reductions result in the lower power consumption and amplifier costs of massive MIMO systems. The significance of overall energy efficiency for upcoming wireless communication systems was discussed by Yang and Marzetta (2013b; 2015), Björnson et al. (2014), Chen et al. (2015), He et al. (2015), and Gozalves (2016). Massive MIMO systems are assisted by beamforming processes to reduce the power consumption of the entire system by computing the optimal quantity of antenna elements that meet several essential criteria for manipulating energy-efficient massive MIMO systems. For each specified power consumption of each BS, the overall energy efficiency is relatively unaffected by the number of working antenna elements in the cell; thus, a common number of working antennas can be implemented for the entire cells in the system to obtain high cost-effectiveness and overall energy efficiency. To meet the terminal throughput requirements, optimization processes for beamforming techniques, such as power control, must be considered to reduce the power consumption at the BS.

2.2.2 Improved spectral efficiency

Power controlling of the uplink and downlink signals, utilization of the information of the training sequence, and improvement of the signal quality by beamforming antenna elements enable capacity improvements. Massive MIMO systems have potential for improving the spectral efficiency of wireless communication systems by installing beamforming antenna arrays with large numbers of serving antenna elements at BSs with coherent precoding and detector processing (Huh et al., 2012; Kim et al., 2013; Ngo and Larsson, 2013; Behjati et al., 2015; Jin et al., 2016; Noh et al., 2016). The spectral efficiencies of cellular systems are influenced by the carrier-to-interference ratio distributions at the mobiles.

A comparison of the performances of an omnidirectional BS and sectored antennas with beamforming in the presence of traffic was presented by Ismail et al. (1999). The performance of traffic in wireless system constructions has been effectively enhanced via the replacement of conventional omnidirectional antenna arrays with dynamic cell sectored construction and beam steering antenna arrays using time division duplex (TDD) procedures. Simulations have shown that the efficiency of dynamic cell sectoring is improved by using beamforming techniques rather than omnidirectional antennas at BSs. Moreover, average beamforming gains increase the power of the downlink signal at the precoder because of the coherent combination of the received signals at all antenna elements. The gain is relative to the download speed; therefore, the data rate can be increased using massive MIMO systems (Pradhan and Roy, 2014; WWRF, 2014; Yan et al., 2014; Liu GY et al., 2016).

2.2.3 Increased system security

The concept of beamforming is to steer the transmitted signal toward the intended user; therefore, the receiver will be the only party to recover the wanted signal from the overlay signal. Physical security can be achieved because the probability of an eavesdropper receiving the transmitted signal will be smaller than when using conventional antennas (Liao et al., 2011; Guo et al., 2015).

2.2.4 Applicability for mm-wave bands

Another advantage of beamforming is that it can be applied to mm-wave bands. Because the majority of the frequency spectrum that is suitable for dense urban cellular communication (e.g., below 5 GHz) is licensed, the only way to increase data rates in the frequency domain is by leveraging the unused frequency bands near the mm-wave range (e.g., 60 GHz and above), as discussed by Cudak et al. (2013), Rappaport et al. (2013), Medbo et al. (2014), and Carton et al. (2015). The main advantage of these frequency bands is their high bandwidth availability. However, the propagation characteristics of these bands are poor, even for short distances.

Highly directive antennas must be used to overcome this limitation. Fortunately, high antenna gains can be achieved at a considerably smaller antenna size because of the high carrier frequency. This means that these directional antennas can also be used with mobile units. However, a fixed narrow beam system is not suitable for mobile applications. This makes beamforming the only viable solution for such applications (Andrews et al., 2014; Roh et al., 2014; Swindlehurst et al., 2014).

2.3 Massive MIMO precoders and detectors

As noted above, massive MIMO systems have the following benefits: enhancement in throughput performance, low-cost components, low power, and efficient energy radiation. Precoding, or pre-equalization of the transmitted signals, is one of the functions involved in MIMO systems and developed for massive MIMO systems that depend on CSI availability to correct the signal errors at BSs. Detectors at terminals (mobile stations) should subsequently recover the desired established signal from antenna array elements at the BS simultaneously in the downlink stage. Detector designs with enhanced power consumption and low estimation complexity are difficult to obtain but extremely important, particularly when the number of antennas increases.

In most cases of massive MIMO systems, mobile stations can precisely follow the instantaneous state of the channel from pilot signals that are characteristically inserted into uplink-transmitted signals from various terminals within the cell; therefore, at the tth (t=1, 2, …, T) time slot, the signals received at the jth BS can be expressed as follows (Hu, 2016):

$${y_{j,{\rm{u}}}}(t) = \sqrt {{P_{\rm{u}}}} \sum\limits_{k = 1}^K {{h_{j,k}}(t){s_k}(t){s_k}(t) + {n_j}(t),} $$
((1))

where \({h_{j,k}}(t) \in {{\mathbb C}^{M \times 1}}\) is the uplink channel vector from the kth user in the cell to the BS, M is the number of elements of the array antenna in the BS, sk(t) is the symbol transmitted by the kth user in the cell at the tth time slot, Pu is the average signal-to-noise ratio (SNR), and \({n_j}(t) \in {{\mathbb C}^{M \times 1}}\) is an additive noise vector received at the tth time slot. At each time slot, linear beamforming is employed at the BS to suppress the interference and enhance the signal. For the kth user in the jth cell, the received signal vector, yj,u(t) in Eq. (1), is processed by the jth BS with beamforming, and the resulting signal is expressed as

$${\tilde s_{j,k}}(t) = w_{j,k}^{\rm{H}}{y_j}(t).$$
((2))

In the downlink stage, the BS deploys N transmitting beamforming antenna elements, and each terminal can be well appointed with multiple beamforming antennas. wj,k denotes the transmit downlink beamforming vector for the kth user in the jth cell. Then, the received signal at the kth user in the jth cell is given by

$${y_{j,k,{\rm{d}}}} = h_{j,j,k}^{\rm{H}}{w_{j,k}}{x_{j,k}} + \sum\limits_{n,l \ne j,k} {h_{n,j,k}^{\rm{H}}} {w_{n,l}}{x_{n,l}} + {n_{j,k}},$$
((3))

where xj,k represents the information signal for the kth user in the jth cell. The signal-to-interference-plus-noise ratio (SINR) at user k is

$${\rm{SIN}}{{\rm{R}}_{j,k}} = {{{{\left| {h_{j,j,k}^{\rm{H}}{w_{j,k}}} \right|}^2}} \over {{{\sum\limits_{n,l \ne j,k} {\left| {h_{n,j,k}^{\rm{H}}{w_{n,l}}} \right|} }^2} + {\sigma ^2}}}.$$
((4))

Recently, detection algorithms with low complexity and high optimal performance have received significant attention from researchers seeking to upgrade conventional wireless communication systems to 5G. We can categorise these precoders/detectors into two main categories: linear precoders/detectors and nonlinear precoders/detectors. Linear signal detectors with low complexity consider all signals, which are transmitted by excluding the desired stream from the specified transmit antenna as interference (Wagner et al., 2012; Kammoun et al., 2014). Thus, interfering signals, which are transmitted from other antennas, are reduced or nullified in the process of detecting desired signals from the specified transmit antenna. Well-known detectors, such as maximum ratio combining (MRC) receivers (also called matched-filter (MF) receivers), zero-forcing receivers (ZFRs), and minimum mean-square-error (MMSE) receivers, are practical candidates for massive MIMO systems.

Using MRC receivers, BSs attempt to obtain the highest SNR (maximum SNR) for every stream and ignore the influence of other multiuser interference. MRC receivers are advantageous in that they simplify signal processing; however, MRCs perform poorly in interference-limited scenarios because they do not address the effects of multiuser interference. In comparison to MRC receivers, ZFRs consider multiuser interference in their calculations; however, they do not consider noise effects.

Through zero-forcing (ZF), multiuser interference can be completely nullified by estimating the orthogonal complement of each stream of the multiuser interference. Yang and Marzetta (2013a) compared the two most well-known linear precoding schemes for massive MIMO (conjugate beamforming precoding and ZF precoding) in terms of power consumption and capacity efficiency. They found that by optimising the management of transmitted power, conjugate beamforming could obtain better overall computational measures compared to ZF precoding because of the larger number of served terminals. Regardless of the computational aspects, conjugate beamforming is more robust than ZF and may thus be preferable. Another advantage of conjugate beamforming is that it results in a situation characterised by decentralised planning, in which each antenna independently possesses its channel estimates.

The main objective of a linear MMSE receiver is reducing the mean-square error of the estimated signal versus the transmitted signal. The performance of massive MIMO systems has been examined and studied from several viewpoints based on the characteristics of various linear receivers. A comparison of the performances of MMSE receivers and MF receivers in realistic system settings was presented by Ju et al. (2013). The results indicated that the MMSE receiver can perform similarly as MF receivers with fewer antennas, particularly under interference conditions. The final formulations for various receivers according to Eqs. (3) and (4) are as follows:

$$\begin{array}{*{20}c} {W = {H^*},\,\,{\rm{for}}\,{\rm{MF,}}} \\ {W = {H^*}{{({H^{\rm{T}}}{H^*})}^{ - 1}},\,\,{\rm{for}}\,\,{\rm{ZF,}}} \\ {W = {H^*}{{\left( {{H^{\rm{T}}}{H^*} + {1 \over {{\rm{SINR}}}}{I_k}} \right)}^{ - 1}},\,\,{\rm{for}}\,\,{\rm{MMSE}},} \\ \end{array} $$

where H* is the Hermitian operation of the channel matrix and Ik is the identity matrix for the kth user. MMSE receivers have performed slightly better than ZFR receivers, but the noise variance must be known at the MMSE receiver. This drawback does not affect conventional MIMO systems but will induce a high complexity for systems with many antennas. Thus, the majority of recent studies have concentrated on ZFRs for massive MIMO systems rather than MMSE receivers (Li and Leung, 2013; He et al., 2014; Jin et al., 2014; Liu DL et al., 2016). Li et al. (2016) evaluated the average sum rate of coordinated scheduling and beamforming of users in a multicell model, and the authors developed low-complexity, multicell, coordinated user scheduling policies for massive MIMO systems. In Le and Kim (2015), spectral efficiency under power scaling laws of massive MIMO systems was analysed, where the authors engaged ZF beamforming and ZF relay for a multipair, massive antenna relaying system.

As noted above, linear detectors are simple but provide poor BER performance. By contrast, nonlinear detectors provide a reasonable BER performance but have a high computational complexity. The best-known nonlinear precoder/detector techniques that can be used for MU MIMO systems are dirty-paper-coding (DPC), vector perturbation (VP), and lattice-aided methods (Masouros et al., 2013). Such detectors can be used to obtain better performances compared to using linear detectors at the cost of higher estimation complexity. The improvement of complexity for nonlinear detectors is the key issue in massive MIMO systems (Mazrouei-Sebdani et al., 2016).

3 Beamforming technique classifications

The beamforming technique is used in smart antennas for transmitting and receiving signals in massive MIMO systems. Smart antennas are antenna arrays with signal processing algorithms that are aware of spatial signal identifiers, such as the direction of arrival (DOA) of the signal, and employ them to evaluate beamforming vectors. These vectors identify and consequently track the desired signal sent from mobile stations. Smart antenna techniques are used particularly in acoustic signal processing, radio astronomy and radio telescopes, track-and-scan radar, as well as in wireless communication systems, such as W-CDMA, UMTS, LTE, and LTE Advanced.

Several methods, such as the multiple signal classification (MUSIC) technique, estimation of signal parameters via rotational invariance techniques (ESPRIT), and the matrix pencil method and its derivatives, have been elaborated as a part of the beamforming technique to predict the DOA of incoming signals and have been implemented via smart antenna systems. Such methods take in spatial spectrum results of the antenna array and compute the DOA based on the peaks of the spectrum (Yang et al., 2010; Liao and Chan, 2011; Oumar et al., 2012; Chuang et al., 2015). Smart antennas can be categorised into three types: diversity, spatial multiplexing, and beamforming. Diversity is used at the transmitting and receiving sides to reduce multipath fading and improve link reliability, whereas spatial multiplexing involves transferring multiple streams of data in parallel to increase the transmission rate.

Several works have been conducted to classify beamforming techniques according to their characteristics. Some scientists classified beamforming techniques in terms of their physical characteristics. Gotsis and Sahalos (2011) classified beamforming techniques into two main categories: switched beamforming and adaptive beamforming. Moreover, they categorised the techniques into various types of array antennas, that is, linear arrays, circular arrays, and rectangular arrays. Another classification of beamforming techniques based on signal processing was presented by Hur et al. (2013) and Bogale and Le (2014), wherein the researchers classified the techniques into analogue beamforming, digital beamforming, and hybrid analogue/digital beamforming. The benefit of employing analogue beamforming is that inexpensive phase shifters are used for massive MIMO systems compared to digital beamforming, which has the advantage of providing more accurate and rapid foundation results to obtain user signals. However, digital beamforming suffers from high complexity and an expensive design; thus, it is not adopted in massive MIMO systems. Hybrid analogue/digital beamforming has been developed for massive MIMO systems to obtain the advantages of analogue and digital beamforming. In addition, many algorithms have provided technological advancements for increasing and optimising the performance of adaptive beamforming antennas.

These algorithms can be classified into two main types: blind adaptive algorithms and non-blind adaptive algorithms. Non-blind adaptive algorithms require known statistics of the transmitted signal, with the objective of determining a weighted path of travel. This objective can typically be achieved using a training signal that is transmitted through the communication link to the terminals to support the detection of the preferred user. By contrast, blind adaptive algorithms do not require any statistical knowledge to be trained; this is clearly expressed by the term ‘blind’. The focus of blind algorithms is on re-establishing some types of physical characteristics of the downlink signal with the goals of maximising the signal to the desired terminal and minimising interference from other terminals. Another factor that can improve the quality-of-service of beamforming techniques is that they can be used for mm-wave bands (wideband) instead of conventional bands (narrowband), that is, 900 MHz-5 GHz. In mm-wave bands, the antenna array is extremely small owing to the size of the wavelength and beam width being extremely sharp, and the maximum range between the BS and the users is a few hundred metres. We present these classifications in Fig. 1.

Fig. 1
figure 1

General beamforming classification

LMS: least-mean-square; RLS: recursive-least-square; SMI: sample matrix inversion; CGA: conjugate gradient algorithm; CMA: constant modulus algorithm; LS-CMA: least square constant modulus algorithm; LCMV: linearly constrained minimum variance; MVDR: minimum variance distortionless response

3.1 Wideband beamforming versus narrowband beamforming

Beamforming can be divided into two categories depending on the signal bandwidth: narrowband beamforming and wideband beamforming. Narrowband beamforming is achieved by an instantaneous linear combination of the received array signals. However, when the involved signals are wideband, an additional processing dimension must be employed for effective operation, such as tapped delay lines (or FIR/IIR filters) or the recently proposed sensor delay lines, which lead to a wideband beamforming system (Liu and Weiss, 2010).

The majority of current wireless communication applications are still focused on narrowband beamforming; however, wideband beamforming becomes an important topic for future wireless communication applications owing to 5G requirements concerning high-frequency band signals for achieving an extremely high data rate. The best example of wideband beamforming that can be implemented for 5G for establishing extremely high speeds and high capacities is mm-wave beamforming. An extensive review of mm-wave (wideband) beamforming is provided in Section 5.

3.2 Switched array beamforming versus adaptive array beamforming

Beamforming schemes are generally classified as either switched-beam systems or adaptive array systems. A switched-beam system depends on a fixed beamforming network that yields established predefined beams. Perhaps the most common solution for fixed beamforming is the Butler matrix, which was developed by Butler and Ralph (1961). A Butler matrix is composed of hybrid couplers, phase shifters, and crossovers. In Ren et al. (2016), the reader can find an all-inclusive review of the Butler matrix functionality and its operation subjects. A switched-beam system requires a switching network, with the objective of choosing a suitable beam to obtain the desired signal from a specific terminal. The majority of the chosen emitted beam might not point to the desired direction. Bae et al. (2014), Huang and Pan (2015), and Tiwari and Rao (2015) addressed these issues (Table 1). Furthermore, a beam typically serves more than one mobile station (Fig. 2a). By contrast, adaptive array systems have the option to formulate a singular beam for each user. This option is realised by weight vectors that are applied via adaptive array processors to detected signals with the objective of controlling phase changes between the elements of the antenna array and their amplitude spreading (Fig. 2b). In this technique, specific beam shapes can be formed, and the directions toward a preferred mobile station are given by the main lobe remote sensing and consequently null toward the interfering sequences.

Fig. 2
figure 2

Adaptive beamforming (a) and switched beamforming (b)

Table 1 Comparison between the two approaches in terms of coverage area, capacity, interference suppression, and complexity

Adaptive beamforming assumes that the BS modernizes the localization of the mobile station. However, accurate localization is a difficult task because a large number of real-time mobile stations may overload the process. Therefore, estimating the DOA of received signals impinging on an antenna array is a main issue for wireless communication systems. It is considerably more difficult to put an adaptive beamforming system into practice than a switched-beamforming system.

By contrast, perfect adaptive beams attempt to reduce the interference between users and achieve considerably improved offered power resources (Huang et al., 2015; Sivasundarapandian, 2015; Wu et al., 2015). Regardless, there are advantages and disadvantages to both categories, which must be considered in implementation for massive MIMO systems. These advantages and disadvantages are shown in Table 1, which illustrates that although adaptive beamforming is difficult to implement, the majority of recent studies and simulations related to massive MIMO prefer this technique to switched beamforming because of its reliability for 5G requirements.

3.3 Adaptive beamforming algorithms: blind algorithms versus non-blind algorithms

In beamforming antenna arrays, signals received at each antenna element of the antenna array are adaptively built up to enhance the overall efficiency of the wireless communication system. The signals detected at the different antennas at the BS pass through multiplication processes with complex weights. The modified weights are immediately summed. In adaptive beamforming antenna arrays, according to the DOA, the estimation beam can be steered to the wanted signal direction, and nullification is applied to unwanted signal directions. The DOA of incoming signals and the direction of interfering signals can be easily estimated by smart antennas. Then, using a beamforming algorithm, the beam of the antenna is created toward the desired signal direction, and the null is formed toward the interfering signal directions. Adaptive beamforming algorithms are categorised into two main types: non-blind adaptive algorithms and blind adaptive algorithms (Arunitha et al., 2015). The general equation for adaptive array output y(t) is given by

$$y(t) = {w^{\rm{H}}}x(t),$$
((5))

where wH denotes the complex conjugate transpose of the weight vector. The weights are computed in an iterative manner based on the array y(t). In non-blind adaptive algorithms, a reference signal is used to adapt the array weights continually. Subsequently, the response of the weights at the end of every iteration is compared to the reference signal, and the produced error signal is implemented to adjust the weights in the algorithms. The error signal is given by

$$e(t) = d(t) - {w^{\rm{H}}}x(t),$$
((6))

where d(t) is the reference signal, which is similar to the original signal, and the algorithm should minimise the error difference between the output signal and reference signal such that the output signal is as close to the original signal as possible. Then, a beam can be formed toward the desired signal and the user can always be tracked.

Some examples of non-blind adaptive algorithms are the least-mean-square (LMS) algorithm, the recursive-least-square (RLS) algorithm (Arablouei and Dogancay, 2012), sample matrix inversion (SMI) (Liu J et al., 2016), and conjugate gradient (CG) (Jamel, 2015). By contrast, blind adaptive algorithms are not dependent on the implementation of a reference signal and, therefore, there is no requirement to adjust the weights of the array.

Famous examples of blind adaptive algorithms include the constant modulus algorithm (CMA) and least square constant modulus algorithm (LS-CMA) (Bhotto and Bajić, 2015). Nulls are designed toward identified interference source directions by adaptively varying the antenna array pattern; hence, the adaptive beamforming technique can operate under interference conditions.

The main generator in adaptive beamforming is the digital signal processor, which deduces the received signals, defines the complex weights, multiplies the produced weights by every element response individually, and modifies the array radiation pattern. The effects of noise signals and interference signals are minimised by the antenna array, which also produces the maximum gain toward the desired direction. Thus, the performance of smart antennas is dependent mainly on the adaptive algorithms that are used for digital beamforming. Most performance criteria of adaptive beamforming algorithms depend on comparisons between various types of algorithms in terms of time of evaluation (speed of conversions and number of iterations) and accurate resolution, which is affected by multipath and interfering signals, to obtain the maximum SNR from a desired terminal.

Recently, two well-known adaptive algorithms were developed based on the null steering approach, LCMV beamforming (Rasekh and Seydnejad, 2014) and MVDR beamforming (Zou and He, 2013); these algorithms use null steering beamforming ideas to produce an adaptive beamforming array. Null steering antennas are critical in wireless communications because they improve the SINR by placing the null in the interference direction while pointing to the main lobe in the desired direction. The null positions in the antenna pattern are functions of the complex weights. Their values are obtained by solving a linear equation. Nevertheless, the direction of the interference signals must be precisely known. This causes a problem in practice because the directions of interference signals vary over time in real-time situations. Adaptive null steering, which can track the interference direction, is adopted to resolve this issue.

Many algorithms have been applied to the adaptive null steering problem. An efficient method based on the bees algorithm was presented by Guney and Onay (2007). One of the main algorithms used for adaptive null steering is the genetic algorithm (GA) (Khan and Tuzlukov, 2011). GAs represent a powerful searching method. The GA proposed in previous works was based on minimising the array output power. The direction of the main lobe must remain in the desired direction during the process.

3.3.1 Linearly constrained minimum variance beamforming

The majority of designed beamforming algorithms require some knowledge of the reference signal and the strength of the desired signal. These limits can be overcome by applying linear constraints to the weight vector. The principal concept of the LCMV beamformer was proposed by Frost (1972), where beamformers of LCMV are spatial filters that select optimum weight vectors to reduce the filter’s response based on power constraints. This condition, together with other constraints, ensures signal protection at the desired location while reducing the variance effects of created signals from other directions. LCMV was developed based on the conventional MMSE for the automatic adjustment of array weights for simplicity.

The main disadvantage of the LCMV technique is its low convergence rate, which makes it inappropriate for application to massive MIMO systems even though it requires only the DOA to maximise the SNR, which simplifies the process estimation. Many different studies have sought to improve the capabilities of LCMV beamforming. Particle swarm optimization, dynamic mutated artificial immune systems, and gravitational search algorithms were incorporated into current LCMV techniques to improve LCMV weights (Darzi et al., 2014). The simulation results proved that the detected SINR of the desired terminals can be improved through the combination of particle swarm optimization (PSO), DM-AIS, and GSA in LCMV based on interference mitigation.

3.3.2 Minimum variance distortionless response beamforming

The main concept of the MVDR beamforming algorithm is to reduce the variance of beam responses by selecting the weights of the antenna element while keeping the gain level that steers the beam to a desired direction constant. The main purpose of the procedure is to terminate strong interference signals from undesired directions. Several methods have been suggested to enhance the validity of the MVDR beamformer, with the most famous and widely used being diagonal loading and its extensions. The initial diagonal loading method was developed by Carlson (1988).

Although many methods based on loading factor evaluation have been proposed, the loading factor remains an issue, and some of these methods suffer from increased algorithm complexity. Vincent and Besson (2004) selected a negative loading level with the objective of maximising the SINR even though steering vector errors were found. A method that depends on the notion of uncertainty sets was recently proposed. Vorobyov et al. (2002) forced the steering vector magnitude responses into a sphere to obtain unity under a non-convex constraint. Gross (2005) proved that this algorithm is from the family of diagonal loading beamformers. Darzi et al. (2016) focused on solving the optimization problem of MVDR. The simulation attempted to demonstrate the cooperation of PSO and gravitational search algorithms to assist MVDR in improving its performance and reducing the effects of interference.

The new algorithm achieved significant improvements over conventional adaptive antennas. Based on 5G requirements that include no latency or accuracy, blind adaptive algorithms are more convenient than non-blind adaptive algorithms for massive MIMO systems. Huang et al. (2015), Sivasundarapandian (2015), and Wu et al. (2015) proposed some blind and semi-blind adaptive algorithms for massive MIMO systems.

However, the aforementioned properties of LCMV and MVDR indicate that MVDR is more suitable for massive MIMO systems than LCMV. Researchers have evaluated the DOA for the aforementioned algorithms in terms of the azimuth angle and elevation angle (2D-DOA), which increases the accuracy of the results, thus reducing inter-cell interference and increasing the performance of the massive MIMO system (Barua et al., 2015; Kiani and Pezeshk, 2015; Shang and Li, 2015; Zhang et al., 2016). In Kiani and Pezeshk (2015), a comparison between several array geometries, including planar arrays and volume arrays, for 2D-DOA estimation using MUSIC is presented. For each geometry, various criteria are taken into consideration and a comparative study of the performance of geometries is carried out. It has been shown that all the proposed geometries have to fit the system performance. However, the accuracy of the estimations reaches its peak by using planar geometries.

Zhang et al. (2016) proposed a new scheme for measuring and estimating the 2D-DOA of propagation paths, called multipath angular estimation, using the array response of PN-sequences (MAPS) in full-dimensional MIMO (FD-MIMO) systems. MAPS has been compared with other algorithms such as MUSIC, ESPRIT, and SAGE by simulations. The results have shown that MAPS outperforms the conventional methods in terms of estimation accuracy and capacity for various SNRs, antenna array sizes, numbers of paths, and delay differences.

3.4 Analogue, digital, and hybrid analogue/digital beamforming

Another classification of beamforming techniques is shown in Fig. 1, where they are categorised into two types: analogue beamforming and digital beamforming. Analogue beamforming was proposed more than 50 years ago. Analogue beamforming antennas are composed of hybrid matrices and fixed phase shifters. The main concept behind analogue beamforming is controlling the phase of each transmitted signal using low-cost phase shifters. A selective radio frequency (RF) switch is used to facilitate the beam steering function (steering angle).

Some modern analogue beamforming antennas have been proposed and offer continuous beamforming. Venkateswaran and van der Veen (2010) proposed analogue beamforming in MIMO communications with phase shift networks. They attempted to cancel interfering signals in the analogue domain and minimise the required ADC resolution. Unfortunately, it is difficult to control the direction of their nulls.

By contrast, digital beamforming consists of many utilities, including DOA estimation, programmable control of antenna radiation patterns, and adaptive steering of its beam and nulls to enhance the SINR (Gross, 2005; Liang et al., 2014; Darzi et al., 2016). These advantages can be obtained only by using digital technology.

The implementation of digital beamforming is not suitable for massive MIMO systems because traditional beamforming is implemented at the baseband, which helps to control the phase and amplitude of the signal; therefore, it requires the carrier frequency of the processed signal be up-converted after a crossover RF chain, which includes digital-to-analogue (D/A) converters, mixers, and power amplifiers. The responses of the RF chain are then combined with the antenna elements. In other words, each antenna array element must be reinforced by a dedicated RF chain. This is expensive to realise in massive MIMO systems because a large number of antenna elements is necessary.

As noted above, analogue beamforming is applied in a simple manner using inexpensive phase shifters. For that reason, analogue beamforming is more cost-effective than digital beamforming. Conversely, analogue beamforming exhibits poorer performance compared to digital beamforming because the amplitudes of the phase shifter are not flexible. To achieve better performance, mixing between digital and analogue beamforming has been proposed and is referred to as hybrid beamforming.

Kim et al. (2013), Haghighat (2014), Bogale et al. (2015), Ying et al. (2015), and Noh et al. (2016) demonstrated that hybrid beamforming concepts, which are a mixture of digital and analogue beamforming, are widely applied to massive MIMO systems. The digital beamforming portion creates baseband signals, whereas the analogue beamforming portion addresses RF chain effects by reducing the number of ADCs/DACs, which improves the outputs of power amplifiers or changes the architecture of the mixers and hence provides cost savings.

Different types of hybrid beamforming systems have been designed and suggested by many researchers. Hur et al. (2013) designed a hybrid beamformer and compared hybrid and digital beamforming in the case of downlink multiuser massive MIMO systems. They investigated the relationship between both digital and hybrid beamforming statistically by varying such factors as the RF chain parameters (ADCs and the number of multiplexed symbols). The simulation results showed that for a certain number of RF chains and ADCs, the difference in performance between digital and hybrid beamforming can be improved by reducing the number of multiplexed symbols. Furthermore, for a particular number of multiplexed symbols, increasing the number of RF chains and ADCs will increase the sum rate of hybrid beamforming that can be obtained.

Recently, researchers have concentrated on designing and developing hybrid beamforming that can operate in mm-wave bands for massive MIMO systems. Chen (2015), Dai et al. (2015a; 2015b), and Ghauch et al. (2016) designed hybrid mm-wave precoding for multiple objectives, such as reducing the weighted sum of squared residuals between the optimal digital beamforming design and hybrid beamforming design. Dai et al. (2015a; 2015b) addressed the problem of channel estimation and hybrid precoding/detectors, and Chen (2015) and Ghauch et al. (2016) reduced the complexity of hybrid beamforming.

4 Millimetre-wave beamforming for massive MIMO systems

Global bandwidth has reached a state of unavailability owing to the rapid growth of wireless communication system requirements. The carrier frequencies of wireless communication systems have ranged between 800 MHz and 5.8 GHz. There is great interest in replacing conventional global bandwidth by wireless mm-wave frequencies because conventional bandwidth currently surpasses the combined bandwidths of all wireless communication systems when it is heavily used.

Millimetre-wave frequencies found some use early on, such as in local multipoint distribution systems (LMDSs) and backhaul services at 20 and 40 GHz, in the 1990s; however, these applications were commercially impractical owing to the high costs of hardware characterising these mm-wave devices.

CMOS production technologies have been developed to obtain substantial reductions in cost and size, thereby creating attractive reasons for mm-wave frequencies to be considered as cost-effective means of mitigating bandwidth issues. Power-efficient circuits, smart antenna arrays, and mm-wave spectrum devices will be established to continue improving the abilities to use mm-wave frequencies as a highly effective solution for future technological development of wireless communication systems.

There are several benefits to using mm-wave frequencies in future 5G networks:

  1. 1.

    A broad spectrum exists at mm-wave frequencies that can be used for current cellular communication systems, including service bandwidth from 28 to 40 GHz, frequency band from 60 to 63 GHz, bandwidths from 71 to 76 GHz, 81 to 86 GHz, and 92 to 95 GHz. However, existing bandwidth has already been fully used and represents a restricted spectrum.

  2. 2.

    At mm-wave wireless communication bands, carrier frequencies can be reused within short distances because of the high debility of signals (attenuation) in free space and penetration.

  3. 3.

    Owing to the small wavelength of mm-wave frequencies, the antennas are sufficiently small, so it becomes practical to construct and implement multipart antenna arrays and integrate them onto boards and chip circuits.

  4. 4.

    Owing to transmission range limitations, in addition to the relatively narrow beam widths of mm-wave communication systems, significant improvements in security and privacy can be obtained.

Although mm-wave systems benefit from a wide range of spectra that can be used, they are power limited because of the large path loss accompanying mm-wave wavelengths because of the high carrier frequency. In addition, such systems are interference limited because of co-channel interference. With the goals of overcoming the uncomplimentary path loss and avoiding co-channel interference, an applicable beamforming scheme for concentrating the transmitted signals into a desired direction is one of the key enablers for wireless mobile communications at mm-wave frequency bands.

However, the small wavelengths of mm-wave bands enable the implementation of a large number of beamforming antenna elements in a compact form to create directional beams that can address the maximum number of terminals. As mm-wave systems have been developed, there has been significant work on beamforming techniques to enhance both spectral efficiency and energy efficiency for massive MIMO systems. The deployment of mm-waves in massive MIMO systems depending on beamforming techniques was explained in detail by Swindlehurst et al. (2014) and Bogale and Le (2015), which discussed mainly the integration of mm-waves and massive MIMO technologies in 5G.

Extensive work is currently being done by researchers to develop an inclusive statistical mm-wave propagation model for channel modelling that can assist in reinforcing next-generation cellular system development. The mm-wave spectrum exhibits strong potential as a compulsory spectrum for 5G wireless systems.

The combination of mm-wave bands and beamforming techniques is of concern in the development of hybrid beamforming techniques and beamforming antenna array design. Roh et al. (2014) developed an mm-wave adaptive beamforming prototype that consists of 32 antenna elements organised in an arrangement of an identical planar array using eight horizontal elements by four vertical elements compressed inside a 60 mm×30 mm prototype. This minor prototype operates at a carrier frequency of 28 GHz. The created beam specifications, which are exclusively based on the full horizontal and vertical width at half maximum of the overall beamforming gain, have been significantly improved.

With the advanced adaptive beamforming system, researchers successfully proved that the mm-wave spectrum can support cells with a coverage radius of a few hundred metres for outdoor and indoor systems with transmit data rates of greater than 500 Mb/s, even in non-line of sight (non-LOS) environments. Sun et al. (2014) proposed principles for adopting the MIMO spatial multiplexing concept in addition to the beamforming concept for upcoming mm-wave wireless communication systems by exploring when either, or both, are most likely to be convenient. Among the contributions of their simulation, they evaluated the performance of general urban steering channel measurements at bandwidths of 28–38 GHz and 73–76 GHz and demonstrated that steering beams regularly occur in the presence of low path loss and slight multipath time dispersion. This means that high link power could be received using beamforming with a simple equalization process. In addition, they proved that digital beamforming is not suitable for mm-wave massive MIMO systems because of RF chains that follow each antenna array element in the system.

Thus, it will be extremely expensive and complex to address digital beamforming with the implementation of large numbers of antennas. Therefore, addressing hybrid beamforming approaches, which are combinations of digital beamforming and analogue beamforming, as shown above, promises to provide enhancements in terms of both power consumption and performance. Li et al. (2014), Huang et al. (2015), Bogale et al. (2016), and Dai and Clerckx (2016) have discussed briefly hybrid beamforming (Table 2). They drew the same conclusion that digital beamforming is not suitable for massive MIMO and proposed some schemes of hybrid beamforming that can avoid digital beamforming implementation problems and improve system performance.

Table 2 Taxonomy of articles based on beamforming classifications

5 Unresolved issues and future trends

Although several research works have focused on beamforming techniques and their applications in massive MIMO systems as a basic concept and realization for 5G systems, there are several issues that have received considerable attention in recent years and must be addressed before implementing massive MIMO systems in practice.

5.1 Pilot contamination

In a massive MIMO system, each terminal specifies an independent pilot sequence to be up-linked. However, the number of these independent pilot sequences is limited; therefore, they must be reused. The effect of reusing the same pilots between different cells will produce a conflict in the antenna array of a BS once they have correlated the desired received pilot signal with the pilot sequence associated with a particular terminal. These associated sequences are referred to as pilot contamination. The array antenna at the BS obtains a channel estimate that is corrupted by a combination of signals from other terminals using the same pilot sequence. These contaminated pilots affect downlink beamforming and result in interference directed toward those terminals, which are using similar pilot sequences. This undesired interference increases with an increasing number of antennas used.

The problem of pilot contamination was defined many years ago; however, its influence on massive MIMO systems appears to be substantially greater owing to the large number of antennas in its construction compared to previous systems (Ngo et al., 2011; Ashikhmin and Marzetta, 2012; Hu et al., 2013; Wang et al., 2013; Yin et al., 2013; de Carvalho et al., 2016).

Various methods have been proposed to alleviate pilot contamination and can be classified into two schemes. One scheme is based on the development of the pilot itself, such as by reducing pilot overhead, or using different frequency reuse factors in multicell systems instead of using a frequency reuse factor of one. This issue was clearly illustrated by Wang et al. (2013), who analysed the effects of pilot contamination using different reuse factors. They performed comparisons of area throughput for different reuse factors, and their results demonstrated that the performance of massive MIMO systems was improved.

By contrast, the other scheme concentrates on the development of precoding at the BS by adjusting the precoding matrix at one BS, which can mitigate the pilot contamination effect. Ahmadi et al. (2016) proposed open-loop power control (OLPC) and pilot sequence reuse schemes that avoid pilot contamination within a group of cells. Scientists compared the performance of a simple least-squares channel estimator with that of the higher-complexity minimum mean square error estimator, and evaluated the performance of the recently proposed coordinated pilot allocation (CPA) technique. They found that for moving users in vehicles, pilot contamination could be efficiently mitigated using both the OLPC and pilot reuse schemes.

Other proposals can also be suggested to mitigate pilot contamination effects without changes in pilot construction and depending on beamforming techniques, such as beamforming optimization methods, which can nullify the effects of inter-cell interference by discriminating the desired channel from the interfered channel in the uplink stage using the 2D angle of arrival (AOA) (Shang and Li, 2015) or based on joint angle and power domain discrimination (Yin et al., 2016).

5.2 Millimetre-wave hybrid beamforming

New categories of hybrid beamforming that are compatible with mm-wave bands for massive MIMO systems are required for the design of beamforming weights compared to lower frequency-band MIMO systems and attempt to reduce the complexity and performance gap between hybrid beamforming and digital beamforming.

5.3 Channel correlation of beamforming array antennas for millimetre-waves

Channel correlation issues represent critical issues in mm-wave applications, particularly when propagation has a habit of being LOS or near LOS and when integrating with massive MIMO systems. Although this would likely avoid the use of maximum ratio combination and maximum ratio transmit precoding for interference suppression, other methods based on beamforming optimizations can be employed and could be extensively studied to mitigate this factor.

5.4 Sparsity of beams

Although cooperation between beamforming techniques and mm-wave systems mitigates many problems and enhances system performance, investigations that have focused on multiuser transmission schemes for mm-wave massive MIMO systems discovered that mm-wave channels exhibit strong sparse propagation after being converted into the angular space because of high path losses. Several studies have been conducted to solve this issue (Yang and Marzetta, 2013b; Lee et al., 2015). A method that depends on a beamforming technique, referred to as beam overlapping, can be used to resolve the sparsity issue. The overlap between the beam patterns intensifies the amount of information carried by the channel outputs, and thus a path can be identified using a combination of multiple channel outputs instead of simply a single channel output.

5.5 Beamforming localization for massive MIMO systems

Although studies on massive MIMO systems have focused extensively on communications, they have also examined the high-accuracy localization. AOA estimation-based beamforming can be used for this purpose. Typically, a source emits a signal, and the AOAs are measured at all BSs using a two-step localization approach; hence, the source location is found by triangulation (Guerra et al., 2015; Garcia et al., 2017). However, the AOA estimates are affected by dense multipath environments, such as urban regions or inside buildings. Novel schemes to incorporate accurate localization based on beamforming techniques must be further developed to realise the benefits of beamforming.

6 Conclusions

This paper presents a comprehensive overview of beamforming techniques in massive MIMO systems. It critically reviews the recent research on various classifications of beamforming techniques and investigates which techniques are more appropriate for use in massive MIMO systems.

Broadband beamforming (mm-wave band beamforming) is more applicable in massive MIMO systems than narrowband beamforming owing to its cost-effective means of mitigating bandwidth issues and its power-efficient circuits in smart antenna array design. Adaptive beamforming is more suitable for massive MIMO systems than switched beamforming because of its ability to eliminate interference and reduce power consumption.

Finally, an optimal beamforming for massive MIMO systems can be achieved by deploying a combination of analogue and digital beamforming (hybrid analogue/digital beamforming) in mm-wave bands combined with optimal algorithms. The optimal algorithms can be one of the adaptive algorithms, such as MVDR, or a combination of two algorithms, to estimate accurately the DOA in terms of the azimuth and elevation angles (2D-DOA). The deployment of such optimal beamforming will provide the highest performance in massive MIMO systems, satisfying the requirements of next-generation wireless communication systems.