A new standardisation and selection framework for real-time image dehazing algorithms from multi-foggy scenes based on fuzzy Delphi and hybrid multi-criteria decision analysis methods


Given the rapid development of dehazing image algorithms, selecting the optimal algorithm based on multiple criteria is crucial in determining the efficiency of an algorithm. However, a sufficient number of criteria must be considered when selecting an algorithm in multiple foggy scenes, including inhomogeneous, homogenous and dark foggy scenes. However, the selection of an optimal real-time image dehazing algorithm based on standardised criteria presents a challenge. According to previous studies, a standardisation and selection framework for real-time image dehazing algorithms based on multi-foggy scenes is not yet available. To address this gap, this study proposes a new standardisation and selection framework based on fuzzy Delphi (FDM) and hybrid multi-criteria analysis methods. Experiments are also conducted in three phases. Firstly, the image dehazing criteria are standardised based on FDM. Secondly, an evaluation experiment is conducted based on standardised criteria and nine real-time image dehazing algorithms to obtain a multi-perspective matrix. Third, entropy and VIKOR methods are hybridised to determine the weight of the standardised criteria and to rank the algorithms. Three rules are applied in the standardisation process to determine the criteria. To objectively validate the selection results, mean is applied for this purpose. The results of this work can be taken into account in designing efficient methods and metrics for image dehazing.


Over the past decades, much attention has been paid to dehazing as evidenced in the increasing number of studies that propose dehazing approaches or investigate the quality of dehazed images [1]. However, determining how to objectively assess the performance of these algorithms remains an open problem that can hinder the development of advanced image restoration methods. There is a fact that the algorithms can be evaluated by a wide range and can also be comparable with each other reliably [2]. The most reliable approach is subjectively assessing quality by human observers. However, this approach is often time-consuming and cannot be integrated into real-time image processing systems. Therefore, an alternative objective quality assessment approach needs to be developed [3].

Various foggy scenes have been made available to test the utility of image dehazing algorithms [4, 5]. Most forms of assessment are equivalent on several foggy scenes [6,7,8,9,10,11,12,13,14]. For example, in [6], the authors considered a variety of evaluation scenes, including inhomogeneous, homogeneous and dark foggy scenes, to test the efficiency of algorithms. However, when evaluating certain algorithms, their efficiency should be tested in consideration of various characteristics and foggy perspectives. Therefore, the advantages and demerits of each algorithm should be considered within each context. Under different hazy scenes, several algorithms can work properly, such as those proposed in [15,16,17]. Therefore, comparing these algorithms from only one perspective is unreasonable.

The efficiency of image dehazing algorithms also needs to be evaluated by using trustworthy approaches [8, 18]. In this case, how several algorithms can be evaluated and how the best algorithm is selected through an effective approach warrant further investigation. From the findings of state-of-the-art image quality assessment (IQA), two concerns need to be addressed. Firstly, determining the best enhanced image and validating the best dehazing results are difficult [4]. For instance, evaluators may not always have the same response regarding the quality of an improved image when using a subjective approach. At the same time, the conventional objective approach cannot effectively solve these problems. Second, [6, 19, 14] have reported that no single defogging algorithm shows an excellent performance across different foggy scenes. Therefore, selecting a defogging algorithm is difficult. The selection and benchmarking of a best image dehazing algorithm based on multiple foggy scenes are therefore identified as the major problems in this research.

Nevertheless, most objective evaluation methods, such as those introduced in [4, 8], use different metrics or criteria to measure the quality of an enhanced image [20]. The diversity of image dehazing criteria enables us to evaluate the performance of image dehazing algorithms from several perspectives. For example, the image visibility (IV) criterion identifies the distortion of hazy images based on edge, contrast and texture information [6], whereas others identify the degree of colour distortion in a hazy image based on the colour restoration (CR) criterion [21]. One requirement for the evaluation and benchmarking processes in image dehazing algorithms is the criterion that can indicate the degree of enhancement presented by a certain algorithm towards a specific type of distortion [4]. However, the ability to determine the best alternative under all conditions of uncertainty must be made obvious to achieve an effective selection process. The set of criteria and their importance can also influence the selection process [22]. As stated in [22,23,24], to evaluate and select the best alternative, the first step is to determine the appropriate criteria. For an objective assessment, several metrics have been proposed in the literature, but the use of these metrics varies from one study to another. At the same time, a model for classifying and recommending the most appropriate measurements is yet to be developed. Therefore, providing a standard image dehazing criteria is crucial to the evaluation and selection of image dehazing algorithms. The standardisation of these criteria presents a challenge for this study.

The objectives of this study are to (1) standardise image dehazing criteria based on the fuzzy Delphi method (FDM), (2) develop a new framework for benchmarking image dehazing algorithms based on hybrid multi-criteria decision analysis methods and (3) validate the proposed framework by using statistical validation methods. The rest of this paper is arranged as follows: ‘Introduction’ defines image dehazing evaluation and benchmarking. ‘Literature Review’ reviews the related studies. ‘Methodology’ reports the methodological standardisation and decision-making steps. ‘Results and Discussion’ illustrates and discusses the results. ‘Validation’ validates the results of the proposed framework, whereas ‘Limitations’ presents the restrictions of this framework. ‘Recommendations for Future Work’ provides several recommendations for future study, and ‘Conclusion’ concludes the paper.

Background and related works

Any new algorithm should be compared according to its perceived quality and time complexity (TC), which are considered main indicators for any comparison scenario. Particularly, image dehazing evaluation is based on quality criteria group from one side and on other side time complexity criteria. In the image dehazing domain, quality refers to the capability of a process to remove unwanted effects from a degraded image and restore its quality back to its original state [2]. However, image dehazing algorithms focus on enhancing the visual quality of a foggy image to make it recognisable by the human eye [25,26,27]. In the image dehazing domain, quality focuses on image visual features instead of signal features. However, under poor weather conditions, the acquired image shows low visibility and colour distortion, both of which can influence analysis and recognition processes [28]. An efficient image defogging algorithm needs not only enhance the visibility, edge and texture information of an image but also preserve its structure and colour [6, 29, 30]. Based on the above scenario and along with [6], evaluating image quality based on dehazing algorithms depends on three sub-criteria, namely IV, CR and image structure similarity (SSIM).

IV is evaluated by measuring the obvious edge, texture information, image contrast and image gradient [6]. Hazy images that can be analysed according to these aspects can be recognised by measuring their degree of visibility. Several metrics have also been employed to indicate the level of enhancement in terms of IV, including blind assessment indicators (e and r) [21], visual contrast measure (VCM) [31] and contrast gain [32] (see “Appendix”). Meanwhile, CR is relevant in retrieving the true colour (amount of lost information) of a certain image that is usually distorted by haze or fog effects. CR measures the rate of saturated pixels after the image defogging process or the similarity of histogram distributions between the foggy and enhanced images [6]. To assess CR performance in an enhanced image, several parameters need to be considered, including the rate of saturated pixels (σ) [21], histogram correlation coefficient (HCC) [33] and colour colourfulness index (CCI) [34] (see “Appendix”). However, an objective assessment of haze removal should consider the dehazing effect and distortions introduced during the haze removal process [5]. Image structure similarity (SSIM) is an evaluation indicator that can only measure the degree of distortion that is caused by the image dehazing process [8]. Although a clear definition of image structure is yet to be formulated, the measures of image structure are strongly correlated with subjective quality ratings, thereby suggesting that high-quality images are closely linked to their original forms in terms of their structure contents (object boundaries) [35]. In general, dehazing algorithms do not change the structural information of an image unless they lead to a serious distortion and edge effect. Moreover, the removal of fog from an image will change the image structure [6]. Several metrics have been used to measure the similarity in the structure of hazy and enhanced images, such as SSIM [36] and the universal quality index (UQI) [37] (see “Appendix”).

The existing methods for image dehazing quality assessment can be classified into visible-edges aware, modulated artificial scenes, comprehensive appraisal models and running speed evaluations [4]. Execution time or TC is an important measure for evaluating the computational complexity of an image dehazing algorithm. The computational cost is determined by calculating the average time spent on a single image in several experiments. By comparing execution time and quality feedback, one can easily determine whether an algorithm can be used as an automatic visual system in real time [4, 10, 38].

Several metrics have also been employed for IQA [39]. According to [6], only 11 metrics have been linked to the 3 previously mentioned criteria. As an extension, this study includes additional metrics and statistics that are related to the criteria previously mentioned in the literature. A total of 74 studies were reviewed to determine how many times these metrics have been used. Some of these studies have focused on the evaluation scenario, whereas others have combined the review and evaluation scenarios. However, most of these studies have developed new algorithms in real-time conditions. Other metrics, including PSNR and MSE, have been excluded because they either fail to specify which type of distortion can be measured or are unable to quantify visual distortion. Those metrics that are only used in underwater IQA are also excluded. Following the aforementioned definitions, these metrics are classified into three groups, and their frequency of usage is specified (see “Appendix”).

As shown in “Appendix,” most of the employed metrics are classified under the visibility criterion given that the main distortion caused by haze is decreasing the visibility of an image. As mentioned above, visibility is evaluated from multiple perspectives, thereby explaining the number of metrics classified under this criterion. Meanwhile, only few metrics have been classified under the other two criteria. In addition, given that a variety of metrics have been adopted in the literature and that most of these metrics lack any clear justification, selecting the most appropriate metric for image dehazing evaluation presents a challenge. In addition, only few studies have considered IV, CR, SSIM and TC altogether as evaluation criteria [6], and some studies have shown differences in how they employ these criteria. Firstly, some studies, such as [7, 40], have only used quality criteria. Secondly, other studies have only measured TC [28, 41, 42]. Thirdly, previous studies have shown differences in how they use quality sub-criteria. For instance, [43] only used SSIM, [44] only used CR and SSIM, [33] used both visibility and CR, and [45, 46] only used visibility. No previous study has used a unified set of criteria in their evaluation process. Also, most of the researchers have used evaluation metric based on their subjective view which have caused a significant conflict to highlight the most influence evaluation criteria. “Appendix” highlights the importance of considering IV, CR, SSIM and TC as criteria in image dehazing evaluation. Nevertheless, whilst TC has no sub-criteria, which can create conflict amongst scholars in their criteria selection process, this study argues that this criterion is important in evaluating image dehazing algorithms. Meanwhile, using any of the other sub-criteria remains a significant challenge despite their frequent usage in the literature given that no previous study has defined the maximum or minimum level of importance of using any metric. Therefore, these image dehazing criteria need to be standardised.

Ishikawa [47] proposed FDM, which incorporates the Delphi method into fuzzy theory. FDM is generally employed when making decisions regarding objective issues. Despite having unclear parameters, the results of FDM are deemed appropriate. FDM also provides a flexible framework that covers many barriers associated with lack of accuracy and clarity. Making decisions with incomplete or inaccurate information creates many problems. Moreover, the decisions made by experts are very subjective and uncertain. Given that uncertainty in this situation is possible and that this type of uncertainty is tailored to the fuzzy set, the data should be expressed in fuzzy numbers instead of absolute ones, and fuzzy sets should be used for analysing expert opinion [48]. The strength of FDM lies in its reduction of the length of the study period by reducing the number of Delphi rounds [49]. Dealing with a fuzzy context involves imprecise descriptions and human linguistics, and employing fuzzy numbers can leave the impression of using an appropriate method for decision making [50]. Nevertheless, FDM is suitable for assessing the importance of the criteria affecting a phenomenon on a highly flexible scale [50, 51]. Furthermore, no shortage of useful information will occur because the membership degree effectively considers all opinions [52]. FDM has been widely used for assessment, standardisation and criteria selection in different domains [24, 47, 52,53,54,55]. In this case, this study employs FDM to standardise image dehazing criteria based on expert opinions.

Numerous criteria have been applied in evaluating algorithms, but selecting the best algorithm remains difficult [4, 6, 19]. These obstacles create problems in MCA [56,57,58,59]. Four practical problems should be considered when selecting the best algorithm, namely the multiple evaluation criteria, criterion importance, criteria trade-off and data variation problems [56, 60,61,62,63]. However, when developing an image dehazing algorithm, the multi-criteria problem that considers only the colour of an image is not enough. Other features, including structure and texture [10], should also be considered in complex scenarios. According to [9] and [11], the evaluation results greatly depend on the selection of metrics or criteria. An effective evaluation and benchmarking scenario is therefore required, and multiple criteria should be considered to define the complexity of hazy scenes because an efficient image dehazing algorithm needs to deal with the characteristics of such scenes.

Multi-criteria decision making (MCDM) is a popular decision-making method and operational research area that addresses the decision criteria problem [64,65,66,67,68,69]. MCDM is also used for structuring, planning and implementing decisions [70,71,72,73,74,75,76]. Given its ability to improve the quality of decisions through a highly reliable and reasonable decision making in contrast to standard procedures, MCDM has been increasingly employed in the literature [77,78,79,80,81]. MDCM has three goals, namely: (1) to assist in the selection of the best possible alternative [82,83,84], (2) to identify the practicable alternative amongst a number of alternatives [85,86,87] and (3) to rank the alternatives in a descending order based on their performance [88,89,90,91,92]. The suitable alternative(s) are given a score [93,94,95]. The basic terms of each MCDM ranking, including the decision matrix (DM) and its criteria, should also be defined [96, 97].

MCDM employs objective and subjective weighting methods [98]. The entropy weighting method is an objective method for criteria weight determination [99]. In the image dehazing domain, a comprehensive evaluation of multiple criteria is carried out by using the entropy weighting method to obtain reliable results [13]. The entropy test calculates the weight of the criterion based on the degree of variation in its values and presents a basis for an exhaustive evaluation [99]. Entropy is a purely monotonous and uncertain function where a lower uncertainty leads to a smaller entropy, and vice versa. Therefore, by measuring entropy values, one can determine the degree of dispersal for each criterion. Increasing the dispersal degree of the index will affect the entire assessment, thereby increasing the weight of the index and leading to highly accurate evaluation results [13].

The VlseKriterijumska Optimizacija I Kompromisno Resenje (VIKOR) method is commonly employed in ranking various alternatives that conform to different criteria. Comparing with technique for order of preference by similarity to ideal solution (TOPSIS) method that does not take into account the comparative value of distances between ideal and negative solution [100]. Therefore, VIKOR is considered the most reasonable approach for addressing real-life issues and ranking alternatives. VIKOR method uses a procedure for compromise priority for numerous response optimisation [101, 102]. The alternatives are initially ranked based on their closeness to the ideal solution, and then, the best alternative is determined [101]. Recent studies have outlined multiple integration instances between VIKOR and entropy to obtain reliable and consistent objective weights [103]. In this case, reliable approaches are adopted. The advantages of both the aforementioned approaches are defined to overcome the uncertainties of a problem [103,104,105,106,107]. In the evaluation and benchmarking of image dehazing algorithms, the integration of entropy and VIKOR is essential. When the weights are allocated to various sub-criteria according to entropy, a foundation for integration is established. However, for ranking image dehazing algorithms, the VIKOR method is recommended.


The adopted methodology is divided into three phases. In the first phase, the image dehazing criteria are standardised and determined by using FDM. In the second phase, the data are presented by performing an evaluation experiment. In the third phase, the weights for the standardised criteria are determined by using the entropy method, and the alternatives are ranked by using VIKOR.


FDM is used to standardise the image dehazing criteria in the following steps:

  • S1 All criteria relevant to image dehazing evaluation are described in the previous section. The selected criteria for FDM are the 26 sub-criteria for IV, CR and SSIM.

  • S2 The number of experts included in FDM is defined as shown in Table 3. These experts are interviewed to determine the importance of the evaluation criteria and to collect their opinions regarding these criteria by disseminating expert opinion forms. Linguistic variables are used in designing these forms.

  • S3 The input data collected from the previous step are transferred to a new form (data fuzzification) and used for further fuzzy data analysis as follows [49]:

    1. 1.

      All linguistic variables (Table 1) are converted into triangular fuzzy numbers (TFN) with values of \(m_{1}\), \(m_{2}\) and \(m_{3}\), where \(m_{1}\) represents the smallest value, \(m_{2}\) represents the most plausible value and \(m_{3}\) represents the maximum value.

      Table 1 Variables for the importance weight of criteria [111]
    2. 2.

      The average value is calculated based on the number of each item and is then divided by the number of experts. The fuzzy numbers are assumed to be rij variables for each criteria for expert \(k\) for \(i = 1, \ldots , m, j = 1, \ldots n, k = 1 \ldots k\) and \(r_{ij} = \frac{1}{k} \left( { \pm r_{1ij} r_{2ij} \pm r_{2ij} } \right)\).

      For every expert, the vertex method is used to calculate the average distance between \(r_{ij}\). The spacing between two fuzzy numbers, \(m = \left( {m_{1} ,m_{2} ,m_{3} } \right)\) and \(n = \left( {m_{1} ,m_{2} ,m_{3} } \right)\), is calculated as

      $$d(\tilde{m}\tilde{n}) = \sqrt {\frac{1}{3} \left[ {(m_{1} - n_{1} )^{2} + (m_{2} - n_{2} )^{2} + (m_{3} - n_{3} )^{2} } \right]}$$

      where \(d\) represents the threshold value of \(m\) and \(n\).

    3. 3.

      According to [108], the first precondition for criteria acceptance is that if the value is less than 0.2, then a consensus has been reached amongst the experts. Meanwhile, the second precondition is that the ratio of expert consensus should be greater than or equal to 75% [109, 110].

  • S4 Average fuzzy numbers are used during the defuzzification process to obtain the fuzzy score (A). The value of fuzzy score (A) must be greater or equal than the mean value (α-cut value) of 0.5 to satisfy the third precondition [112, 113]. The following equations can be used to obtain the fuzzy score (A) [49]:

    $$A_{max} = \frac{1}{3}*\left( {m_{1} + m_{2} + m_{3} } \right)$$
    $$A_{max} = \frac{1}{4}*\left( {m_{1} + m_{2} + m_{3} } \right)$$
    $$A_{max} = \frac{1}{6}*\left( {m_{1} + m_{2} + m_{3} } \right)$$
  • S5 The value of fuzzy score (A) can be used as a determinant and priority for an element according to expert opinions. The elements are ranked according to the average fuzzy score. The ranking can help decide whether certain objects should be preserved or discarded [51].

Multi-perspective DM

The multi-perspective DM is an essential part of the proposed framework for the standardisation and selection of image dehazing algorithms. This matrix comprises decision alternatives and evaluation criteria based on multiple foggy perspectives. In addition to the scenario, the evaluation criteria in DM include the main and sub-criteria, which are used to measure the quality and TC of image dehazing algorithms from three perspectives, namely inhomogeneous, homogeneous and dark foggy scenes. In other words, a user can benchmark real-time image dehazing algorithms based on these perspectives simultaneously through the proposed DM to determine the best algorithm. The DM uses the experiment data extracted from the LIVE Image Defogging Database [114] to evaluate nine algorithms, namely Dehazenet [115], MSCNN [116], Colores et al. [117], Zhu [118], multi-band [119], CO-DHWT [120], Meng et al. [121], Liu et al. [122] and Berman et al. [123]. The data are generated from the crossover between nine algorithms and the image dehazing sub-criteria that are defined based on the literature (TC) and standardisation process (quality sub-criteria) by FDM. These data, along with the identified sub-criteria, are used to evaluate each image dehazing algorithm. The complete data of this matrix are presented in Sect. 4.2. Nevertheless, to mention our evaluation criteria, algorithms were yielded by MATLAB 2018a on a personal computer with Windows 10 operating system, Intel Core (TM) i7, RAM 8 GB.

Table 2 illustrates the multi-perspective DM, who values are obtained from the evaluation of the quality and TC of nine image dehazing algorithms.

Table 2 Multi-perspective DM

Hybrid entropy–VIKOR

To develop a procedure for selecting the best real-time image dehazing algorithm based on multi-perspective DM, a hybrid entropy–VIKOR method is introduced, in which the weight of the criterion from the entropy method is amalgamated with the other steps of VIKOR (Fig. 1). VIKOR is applied to address the practical problems related to the (1) multiple evaluation criteria for each perspective, and the (2) trade-off and conflicting issues experienced by the proposed DM. Meanwhile, entropy is utilised to find the weights of the criteria and to determine (3) the importance of the criteria used by the proposed DM. The steps are summarised as follows:

Fig. 1

Standardisation and selection framework for real-time image dehazing algorithms

Entropy weights for standardised criteria

Based on the evaluation data in Sect. 3.2, the weights of various standardised criteria are determined as follows by using the entropy method [124]:

  • S1 Normalise the evaluation criteria as

    $$r_{ij} = \frac{{x_{ij} }}{{\mathop \sum \nolimits_{i = 1}^{m} x_{ij} }}$$

    A DM of the multi-criteria problem with m alternatives and n criteria, where \(x_{ij} = \left( {i = 1, 2, \ldots , m; j = 1, 2, \ldots , n} \right)\), shows the performance value of the \(i\)th alternative to the \(j\)th standardised criteria.

  • S2 For each standardised criterion, the entropy values \(e_{j}\) are calculated as

    $$e_{j} = - h\mathop \sum \limits_{j = 1}^{m} r_{ij} . \ln r_{ij} ,j = 1, 2, \ldots ..n$$

    where \(h\) is the entropy constant and is equal to \(\left( {\ln m} \right)^{ - 1}\), and \(r_{ij} .\ln r_{ij}\) is equal to 0 if \(r_{ij} = 0\) [125].

  • S3 Define the divergence of each criterion as

    $$d_{j } = 1 - e_{j} , j = 1, 2, \ldots ..n$$

    A higher \(d_{j }\) indicates the greater importance of the \(j\)th criterion.

  • S4 Determine the weight of each standardised criterion as

    $$w_{j} = \frac{{d_{j} }}{{\mathop \sum \nolimits_{j = 1}^{n} d_{j} }},\quad j = 1, 2, \ldots ..n$$

    where \(w_{j}\) is the degree of importance of criteria \(j\).

A lower entropy value corresponds to a greater entropy weight, which in turn suggests that a specific criterion provides more information and is more significant than other decision-making criteria [125].

VIKOR for ranking real-time algorithms

In the decision-making process, the weighted matrix is used as the basis for ranking the available alternatives. In Sect. 3.3.1, standardised criteria weights are obtained and applied for each criterion in the DM to obtain a weighted DM. Based on this weighted DM, the real-time image dehazing algorithms are assessed and ranked. The ranking process is described as follows [126]:

  • Step 1 Identify the best \(f^{ *} i\) and worst \(f^{ - } i\) values of all criterion functions, \(i = 1, 2, \ldots ,n\). If the \(i^{th}\) function represents a benefit, then

    $$f_{i}^{*} = \mathop {\hbox{max} }\limits_{j} f_{ij} ,f_{i}^{ - } = \mathop {\hbox{min} }\limits_{j} f_{ij} ,$$

    where \(f_{ij}\) is the value of the \(i^{th}\) criterion function for alternative \(x_{i}\). The ideal solution maximises the benefit criteria and minimises the cost criteria, whereas the negative ideal solution maximises the cost criteria and minimises the benefit criteria. The so-called benefits criteria are the maximisation criteria, whereas the cost criteria are the minimisation criteria [127].

  • Step 2 Calculate the criteria weights based on entropy. A set of weights \(w = w_{1} , w_{2} , w_{3 } , \ldots ,w_{j} , \ldots , w_{n}\) is accommodated in the DM and is equal to 1. The corresponding matrix can be determined as

    $$WM = wi*\frac{{f^{*} i - fij }}{{f^{*} i - f^{ - } i }}$$

    which produces the following weighted matrix:

    $$\left[ {\begin{array}{*{20}l} {w_{1} (f^{*} 1 - f11)/\left( {(f^{*} 1 - f^{ - } 1} \right)} \hfill & { w_{2} (f^{*} 2 - f12)/\left( {(f^{*} 2 - f^{ - } 2} \right)} \hfill & \ldots \hfill & {w_{i} (f^{*} i - fij)/\left( {(f^{*} i - f^{ - } i} \right)} \hfill \\ {w_{1} (f^{*} 1 - f21)/\left( {(f^{*} 1 - f^{ - } 1} \right)} \hfill & { w_{2} (f^{*} 2 - f12)/\left( {(f^{*} 2 - f^{ - } 2} \right)} \hfill & \ldots \hfill & {w_{i} (f^{*} i - fij)/\left( {(f^{*} i - f^{ - } i} \right)} \hfill \\ \vdots \hfill & \vdots \hfill & \vdots \hfill & \vdots \hfill \\ {w_{1} (f^{*} 1 - f31)/\left( {(f^{*} 1 - f^{ - } 1} \right)} \hfill & {w_{2} (f^{*} 2 - f12)/\left( {(f^{*} 2 - f^{ - } 2} \right)} \hfill & \ldots \hfill & {w_{i} (f^{*} i - fij)/\left( {(f^{*} i - f^{ - } i} \right)} \hfill \\ \end{array} } \right]$$
  • Step 3 Calculate the Sj and Rj values and j = 1, 2, 3,….,m, i = 1, 2, 3,…,n as

    $$Sj = \mathop \sum \limits_{i = 1}^{n} w*\frac{{f^{*} i - fij }}{{f^{*} i - f^{ - } i }}$$
    $$Rj = \mathop {\hbox{max} }\limits_{i} wi *\frac{{f^{*} i - fij }}{{f^{*} i - f^{ - } i }}$$

    where \(Sj\) and \(Rj\) denote the utility and regret measures for alternative \(f_{i}\), respectively, and \(w_{i}\) specifies the relative weights of the criterion.

  • Step 4 Compute \(Q_{j}\) and \(j = \left( {1,2, \cdots ,j} \right)\) by using the following relation:

    $$Q_{\text{j}} = \frac{{{\text{v}}\left( {S_{\text{j}} - S^{*} } \right)}}{{S^{ - } - S^{*} }} + \frac{{\left( {1 - {\text{v}}} \right)\left( {R_{\text{j}} - R^{*} } \right)}}{{R^{ - } - R^{*} }}$$


    $$\begin{aligned} S^{*} = \mathop {\hbox{min} }\limits_{j} S_{j } ,S^{ - } = \mathop {\hbox{max} }\limits_{j} S_{j } \hfill \\ R^{*} = \mathop {\hbox{min} }\limits_{j} R_{j } ,R^{ - } = \mathop {\hbox{max} }\limits_{j} R_{j } \hfill \\ \end{aligned}$$

    v is presented as the strategy weight of ‘the majority of the criteria’ (or ‘the maximum group utility’), where \(v = 0.5\).

  • Step 5 Rank the alternatives based on \(Q_{\text{j}}\). A lower \(Q_{\text{j}}\) indicates a better alternative. In other words, the alternative (\(a^{\prime}\)) is identified as the best by the measure \(Q\) (minimum) if the following rules are satisfied:R1. ‘Acceptable advantage’

    $$Q(a^{\prime\prime}) - Q(a^{\prime}) \ge DQ$$

    where (\(a^{\prime\prime}\)) is the alternative ranked second according to \(Q\), \(DQ = 1/\left( {j - 1} \right)\), and \(j\) is the number of alternatives.R2. ‘Stability’ is acceptable within the decision-making context. Alternative \(a^{\prime}\) should also be determined as the best by \(S\) and/or \(R\). This compromise solution is stable within the decision-making process and can be treated as ‘voting by majority rule’ \((v > 0.5)\), ‘by consensus’ \(\left( {v \cong 0.5} \right)\) or ‘with veto’ (\(v < 0.5\)), where v represents the decision-making strategy weight of ‘the majority of criteria’ (or ‘the maximum group utility’). The \(Q\) value indicates that a certain algorithm has higher evaluation criteria values compared with the other algorithms.

Results and discussion

This section presents the findings of the proposed standardisation and selection framework. Section 4.1 presents the standardisation results based on FDM, Sect. 4.2 presents the results of multi-perspective DM, Sect. 4.3 presents the entropy results, and Sect. 4.4 presents the VIKOR results.

FDM results

Different decision makers have varying objectives and expectations, and their judgment is influenced by the criteria for evaluating image dehazing algorithms from different perspectives. According to [128], there is no limit to the number of experts who can participate in FDM. Meanwhile, [129] suggested that 8 to 12 experts is enough only if they have homogeneous backgrounds. Nevertheless, previous studies typically employ 3 to 15 experts [130], whereas others have employed 16 [131], 20 [132] and 17 experts [50]. Therefore, a panel of 16 experts, which is within the recommended range, can be considered adequate for this study. According to [108], if the acceptable value of d is d < 0.2, then a consensus is achieved amongst the experts. Specifically, if the percentage of agreement amongst experts m × n is greater than 75% [109], then one can proceed with the other FDM steps. Otherwise, a second round of FDM should be conducted. Data were collected from these experts after two rounds. The first round involved 20 experts from different organisations and with different backgrounds, but their percentage of agreement on item agreeability was only 68% (less than 75%). In this case, a second round of FDM should be performed. The experts’ responses were gathered by interviewing them and by scaling their responses on hard copies of experts’ forms. Afterwards, by using information obtained from Google Scholar, ResearchGate and official university websites, links leading to the experts’ form were sent to those experts based overseas or those who prefer to answer this form online instead of receiving a hard copy.

By revealing their background, previous studies were categorised into three domains, namely image processing, image dehazing and IQA. As shown in Table 3, most of the feedbacks were obtained from decision makers with experience in image dehazing and processing and from some experts in the image quality domain. Most of these experts have work experience of over 15 years, with some only having 6–10 years of experience. Compared with those from Iraq, the experts from the USA, UK and China were from universities or organisations based on Malaysia.

Table 3 Experts panel

To apply fuzzy operations on the input data, the data collected from the 16 target experts were converted from linguistic forms into crisp and fuzzy numbers (Table 4). The average minimum value (m1), most appropriate value (m2) and maximum value (m3) should be considered in each reported answer. TFN aims to illustrate the fuzziness or vagueness in an expert’s opinion. Each opinion had a certain amount of uncertainty that could not be measured by using a Likert scale given its fixed score. An object called ‘CNR’ was assumed to be given a score of 5 (‘strongly agree’) by an expert. The score is translated into the lowest, most rational and the maximum 0.6, 0.8 and 1.0 ratings. It specified the expert’s agreement to the element is 60%, 80% and 100%, respectively.

Table 4 D value condition results

To check whether a certain criterion is suitable, three preconditions should be satisfied. The d-value specifies the acceptability of a criterion based on the consensus amongst experts. By finding the difference between the average fuzzy number and the fuzzy number given by each expert, the d-value for each criterion can be determined. The post-data analysis results in Table 5 show that all sub-criteria have satisfied the first precondition of acceptability (d-value ≤ 0.2), except for EBCM, which has obtained a d-value of 0.212. Therefore, this sub-criterion was discarded given the lack of consensus amongst experts. A total of 25 sub-criteria remained.

Table 5 Expert agreement and average fuzzy score

The second precondition for item acceptability is to achieve an agreement percentage of no lower than 75%. Meanwhile, the third precondition is the defuzzification process, where each fuzzy number is converted into a crisp number by obtaining the score and averaging the fuzzy number of each sub-criteria. However, same sub-criteria have included in calculation the percentage of agreement. Table 6 shows the results for these preconditions.

Table 6 Summary of findings

As shown in Table 5, the percentages of agreement on the image dehazing sub-criteria are 63%, 69%, 88% and 94%, respectively. In other words, the majority of the sub-criteria, including IVM, VCM, STD, AG, IE, GCF, AMPL, Loss, CNR, VSNR, WSNR, Sharpness, VIF, CIEDE2000s, CEF, CCI, MS-SSIM and IW-SSIM, did not reach enough agreement amongst the experts (less than 75%). In this case, these criteria were discarded along with EBCM. By contrast, e, \(\bar{r}\), CG, ∑, HCC, SSIM and UQI received adequate agreement amongst the experts (more than 75%). Most of these sub-criteria scored 94%, with only HCC receiving an agreement percentage of 88%. However, unlike the first round of FDM, the second round achieved an agreement percentage of 75%, thereby confirming that the sub-criteria were accepted by all experts. Only seven sub-criteria successfully satisfied the second precondition of item acceptability.

A defuzzification analysis was performed to check whether the sub-criteria satisfied the third precondition. As shown in Table 5, most of these sub-criteria failed to score 0.5 or above. Only e, \(\bar{r}\), CG, ∑, HCC, SSIM and UQI successfully satisfied this precondition with scores of greater than 0.5 except for HCC, which scored exactly 0.5. This result again confirms the consistency of results for these seven sub-criteria. In the last step of FDM, the target experts ranked the sub-criteria according to their average fuzzy numbers. As shown in Table 7, each of those sub-criteria that satisfied all the aforementioned preconditions were given high rankings by the experts. UQI ranked the highest, followed by SSIM, ∑, e, \(\bar{r}\), CG and HCC. Meanwhile, the 19 sub-criteria that failed to meet the preconditions were given low rankings. On the whole, the experts have given high priority to SSIM than to IV and CR.

Table 7 DM data

Table 6 summarises the FDM results and the achievement of the three preconditions.

Only seven sub-criteria, namely e, \(\bar{r}\), CG, ∑, HCC, SSIM and UQI, were accepted by the panel of experts for evaluating image dehazing algorithms, whereas the other 19 sub-criteria were rejected.

Multi-perspective DM data

Based on the evaluation data mentioned in Sect. 3.2 and the standardised sub-criteria in Sect. 4.1, the data were generated from the crossover between algorithms and standardised sub-criteria. Table 7 presents the completed DM, wherein nine real-time algorithms are evaluated based on eight evaluation criteria from three evaluation perspectives.

Entropy weighting results

Based on the DM data in Sect. 4.2, weighting was performed objectively for eight criteria for each foggy scene as shown in Table 8.

Table 8 Entropy values and weights for standardised criteria

The entropy values and weights in Table 8 are obtained by using Eqs. (5)–(8). In the three foggy scenes, HCC and UQI achieved the maximum and minimum entropy weights, respectively. The highest entropy weight criteria were considered the key criteria, whereas the lowest entropy weight criteria were deemed unimportant.

VIKOR ranking

The real-time image dehazing algorithms were ranked based on the multi-perspective weighted DM, which was obtained by using Eqs. (9) and (10). e, r, CG and HCC were selected as the benefit criteria, whereas Σ, SSIM, UQI and TC were selected as the cost criteria. By using Eqs. (11) and (12), the distances of the alternatives from the positive and negative ideal solutions were determined. Equation (13) was used to calculate the Qi values of nine real-time image dehazing algorithms. These algorithms were then ranked, and the optimal one was selected based on VIKOR. The results are shown in Table 9.

Table 9 VIKOR ranking results

MSCNN outranks the other algorithms, which have obtained minimum values for Si, Ri and Qi. Therefore, MSCNN was selected as the optimal real-time image dehazing algorithm. The ranking shown in the above table can be considered the final ranking results that will serve as the basis of the validation.

Validation procedure

Ranking real-time image dehazing algorithms is difficult because they depend on multiple conflicting criteria. The results of the framework were validated by using the objective approach proposed in [60, 61, 126]. Following [133], the real-time image dehazing algorithms were classified into several groups, and the results for each group were expressed in mean values to validate the rankings provided by the proposed framework. The mean for each group was calculated as follows by dividing the total perceived results by the amount of results:

$$\bar{x} = \frac{1}{n}\mathop \sum \limits_{i = 1}^{n} x_{i}$$

where \(x_{i}\) = all x-values; n number of items.

The first group of algorithms obtained a lower mean compared with the two other groups. Meanwhile, the second group obtained a mean that was lower or equal to that of the third group yet was higher than that of the first group. Meanwhile, the mean of the third group was higher than that of the first group and is equal to that of the second group. The variance in the mean values of the selected image dehazing algorithms ensures that the results are consistently (systematically) ranked [60, 61, 126]. Table 10 presents the results after the normalisation and weighting of the raw data of the three groups.

Table 10 Validation results

Table 10 presents the validation results. Obviously, the first group had a lower mean than the second group, whilst the mean of the second group was lower than that of the third group. Therefore, the ranking of the real-time image dehazing algorithms is validated, and the algorithms are systematically ranked.


The main contribution of this paper lies in its development of a framework for standardising evaluation criteria based on FDM and its selection of an optimal real-time image dehazing algorithm based on hybrid MCDM methods from multi-foggy scenes. A total of 6 main criteria and 26 sub-criteria were identified based on the literature review. The classification and usage frequency of these criteria were reviewed, and the sub-criteria were categorised into three groups. The majority of the 26 sub-criteria were classified under IV. The e, \(\bar{r}\), ∑ and SSIM sub-criteria have been widely used in the literature. FDM was employed to standardise the sub-criteria according to expert opinions. All 26 sub-criteria must satisfy the three preconditions of FDM. e,\(\bar{r}\), CG, ∑, HCC, SSIM, UQI were used as criteria in the multi-perspective DM. The optimal real-time image dehazing algorithm was selected based on three foggy scenes. The processes and steps of the proposed framework were also outlined. The multi-perspective DM was constructed based on the crossover between the standardised criteria and the nine real-time image dehazing algorithms. The selection procedure was formulated according to the proposed hybrid entropy–VIKOR method. The final weights obtained from the entropy method highlighted the importance of each image dehazing criteria based on three foggy scenes. VIKOR was adopted to rank and select the best real-time image dehazing algorithm according to the quantitative information of the measured criteria. The results reveal that (1) FDM effectively solves the challenges in the standardisation of image dehazing criteria, (2) the hybridisation of the entropy and VIKOR methods can effectively solve the challenges in the selection of the optimal image dehazing algorithm, (3) 19 sub-criteria have failed to satisfy the fuzzy Delphi constraints, and e,\(\bar{r}\), CG, ∑, HCC, SSIM and UQI have successfully satisfied the acceptability preconditions, (4) the rankings of real-time image dehazing algorithms obtained from VIKOR identify MSCNN as the best algorithm, and (5) the ranking results are valid. Several technical points need to be addressed in future studies. Specifically, these studies should confirm the contributions of this work by conducting objective experiments. Additional criteria also warrant further examination and need to be included in the multi-perspective DM. However, the standardised criteria can be used in any evaluation and benchmarking scenario in the image dehazing domain. The observations presented in this work may also be considered when designing a new image dehazing metric dedicated to evaluate the performance of an algorithm.


  1. 1.

    Liu S, Rahman M, Wong C, Lin S, Jiang G, Kwok N (2015) Dark channel prior based image de-hazing: a review. In: 2015 5th international conference on information science and technology (ICIST). IEEE, pp 345–350

  2. 2.

    El Khoury J, Le Moan S, Thomas J-B, Mansouri A (2018) Color and sharpness assessment of single image dehazing. Multimed Tools Appl 77(12):15409–15430

    Google Scholar 

  3. 3.

    Hu B, Li L, Liu H, Lin W, Qian J (2019) Pairwise-comparison-based rank learning for benchmarking image restoration algorithms. IEEE Trans Multimed 21(8):2042–2056

    Google Scholar 

  4. 4.

    Zhu Q, Hu Z, Ivanov K (2015) Quantitative assessment mechanism transcending visual perceptual evaluation for image dehazing. In: 2015 IEEE international conference on robotics and biomimetics (ROBIO), 6–9 Dec 2015, pp 808–813. https://doi.org/10.1109/robio.2015.7418869

  5. 5.

    Hu ZY, Liu Q (2014) A method for dehazed image quality assessment. In: Wen Z, Li T (eds) Practical applications of intelligent systems, Iske 2013. Advances in intelligent systems and computing, vol 279, pp 909–913

  6. 6.

    Xu Y, Wen J, Fei L, Zhang Z (2016) Review of video and image defogging algorithms and related studies on image restoration and enhancement. IEEE Access 4:165–188. https://doi.org/10.1109/ACCESS.2015.2511558

    Article  Google Scholar 

  7. 7.

    Wang W, Yuan X (2017) Recent advances in image dehazing. IEEE/CAA J Autom Sin 4(3):410–436. https://doi.org/10.1109/JAS.2017.7510532

    MathSciNet  Article  Google Scholar 

  8. 8.

    Mai J, Zhu Q, Wu D (2014) The latest challenges and opportunities in the current single image dehazing algorithms. In: 2014 IEEE international conference on robotics and biomimetics (ROBIO 2014), 5–10 Dec 2014, pp 118–123. https://doi.org/10.1109/robio.2014.7090317

  9. 9.

    Guo F, Tang J, Cai ZX (2014) Objective measurement for image defogging algorithms. J Cent South Univ 21(1):272–286. https://doi.org/10.1007/s11771-014-1938-z

    Article  Google Scholar 

  10. 10.

    Liu X, Hardeberg JY (2013) Fog removal algorithms: survey and perceptual evaluation. Eur Workshop Vis Inf Process (EUVIP) 10–12(2013):118–123

    Google Scholar 

  11. 11.

    Hsieh CH, Horng SC, Huang ZJ, Zhao Q (2017) Objective Haze removal assessment based on two-objective optimization. In: 2017 IEEE 8th international conference on awareness science and technology (iCAST), 8–10 Nov 2017, pp 279–283. https://doi.org/10.1109/icawst.2017.8256463

  12. 12.

    Chengtao C, Qiuyu Z, Yanhua L, IEEE (2015) A survey of image dehazing approaches. In: 2015 27th Chinese control and decision conference, pp 3964–3969

  13. 13.

    Wang K, Wang H, Li Y, Hu Y, Li Y (2018) Quantitative performance evaluation for dehazing algorithms on synthetic outdoor hazy images. IEEE Access 6:20481–20496

    Google Scholar 

  14. 14.

    Li B et al (2019) Benchmarking single-image dehazing and beyond. IEEE Trans Image Process 28(1):492–505

    MathSciNet  MATH  Google Scholar 

  15. 15.

    Zuiderveld K (1994) Contrast limited adaptive histogram equalization. In: Graphics gems IV. Academic Press Professional, Inc., pp 474–485

  16. 16.

    Petro AB, Sbert C, Morel JM (2014) Multiscale retinex. Image Processing On Line, pp 71–88

  17. 17.

    Santra S, Chanda B (2016) Day/night unconstrained image dehazing. In: 2016 23rd international conference on pattern recognition (ICPR). IEEE, pp 1406–1411

  18. 18.

    Jiang X, Sun J, Ding H, Li C (2018) Video image de-fogging recognition algorithm based on recurrent neural network. IEEE Trans Ind Inform 14(7):3281–3288. https://doi.org/10.1109/tii.2018.2810188

    Article  Google Scholar 

  19. 19.

    Ma KD, Liu WT, Wang Z, IEEE (2015) Perceptual evaluation of single image dehazing algorithms. In: 2015 IEEE international conference on image processing, ICIP. IEEE, pp 3600–3604

  20. 20.

    Senthilkumar K, Sivakumar P (2019) A review on haze removal techniques. In: Peter JD, Fernandes SL, Thomaz CE, Viriri S (eds) Computer aided intervention and diagnostics in clinical and medical images. Springer, Berlin, pp 113–123

    Google Scholar 

  21. 21.

    Hautière N, Tarel J-P, Aubert D, Dumont E (2011) Blind contrast enhancement assessment by gradient ratioing at visible edges. Image Anal Stereol 27(2):87–95

    MathSciNet  MATH  Google Scholar 

  22. 22.

    Jafari A, Jafarian M, Zareei A, Zaerpour F (2008) Using fuzzy Delphi method in maintenance strategy selection problem. J Uncertain Syst 2(4):289–298

    Google Scholar 

  23. 23.

    Khatami Firoozabadi A, Bamdad Soofi J, Taheri F, Salehi M (2009) Presentation decision support system inconjunction with supplier selection and evaluation using the UTA method. J Manag Dev 13–88

  24. 24.

    Sultana I, Ahmed I, Azeem A (2015) An integrated approach for multiple criteria supplier selection combining Fuzzy Delphi, Fuzzy AHP and Fuzzy TOPSIS. J Intell Fuzzy Syst 29(4):1273–1287

    MathSciNet  MATH  Google Scholar 

  25. 25.

    Kumari A, Sahoo SK (2015) Fast single image and video deweathering using look-up-table approach. AEU Int J Electron Commun 69(12):1773–1782. https://doi.org/10.1016/j.aeue.2015.09.001

    Article  Google Scholar 

  26. 26.

    Sun W, Wang H, Sun CH, Guo BL, Jia WY, Sun MG (2015) Fast single image haze removal via local atmospheric light veil estimation. Comput Electr Eng 46:371–383. https://doi.org/10.1016/j.compeleceng.2015.02.009

    Article  Google Scholar 

  27. 27.

    Pal NS, Lal S, Shinghal K (2018) Visibility enhancement of images degraded by hazy weather conditions using modified non-local approach. Optik 163:99–113. https://doi.org/10.1016/j.ijleo.2018.02.067

    Article  Google Scholar 

  28. 28.

    Rong W, XiaoGang Y (2012) A fast method of foggy image enhancement. In: Proceedings of 2012 international conference on measurement, information and control, vol 2, 18–20 May 2012, pp 883–887. https://doi.org/10.1109/mic.2012.6273428

  29. 29.

    Pan XX, Xie FY, Jiang ZG, Shi ZW, Luo XY (2016) No-reference assessment on haze for remote-sensing images. IEEE Geosci Remote Sens Lett 13(12):1855–1859. https://doi.org/10.1109/lgrs.2016.2614890

    Article  Google Scholar 

  30. 30.

    Elhefnawy EI, Ali HS, Mahmoud II, (2016) Effective visibility restoration and enhancement of air polluted images with high information fidelity. In: ElKhamy S, ElBadawy H, ElDiasty S (eds) 2016 33rd National radio science conference, NRSC, pp 195–204

  31. 31.

    Jobson DJ, Rahman Z-U, Woodell GA, Hines GD (2006) A comparison of visual statistics for the image enhancement of foresite aerial images with those of major image classes. In: Rahman Z, Reichenbach SE, Neifeld MA (eds) Visual information processing XV, vol 6246. International Society for Optics and Photonics, Bellingham, p 624601

    Google Scholar 

  32. 32.

    Economopoulos TL, Asvestas PA, Matsopoulos GK (2010) Contrast enhancement of images using partitioned iterated function systems. Image Vis Comput 28(1):45–54

    Google Scholar 

  33. 33.

    Zhang E, Lv K, Li Y, Duan J (2013) A fast video image defogging algorithm based on dark channel prior. In: 2013 6th International congress on image and signal processing (CISP), vol 01, 18 Dec 2013, 18 Dec. 2013, pp 219–223. https://doi.org/10.1109/cisp.2013.6743990

  34. 34.

    Guo F, Peng H, Tang J (2016) Fast defogging and restoration assessment approach to road scene images. J Inf Sci Eng 32(3):677–702

    Google Scholar 

  35. 35.

    Larson EC, Chandler DM (2010) Most apparent distortion: full-reference image quality assessment and the role of strategy. J Electron Imaging 19(1):011006

    Google Scholar 

  36. 36.

    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Google Scholar 

  37. 37.

    Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Process Lett 9(3):81–84

    Google Scholar 

  38. 38.

    Perez J, Sanz PJ, Bryson M, Williams SB (2017) A benchmarking study on single image dehazing techniques for underwater autonomous vehicles. In: OCEANS 2017—Aberdeen, 19–22 June 2017, pp 1–9. https://doi.org/10.1109/oceanse.2017.8084771

  39. 39.

    Kim K, Kim S, Kim K-S (2017) Effective image enhancement techniques for fog-affected indoor and outdoor images. IET Image Proc 12(4):465–471

    Google Scholar 

  40. 40.

    Sadhvi N, Kumari A, Sudha TA (2016) Bi-orthogonal wavelet transform based single image visibility restoration on hazy scenes. In: 2016 International conference on communication and signal processing (ICCSP), 6–8 April 2016, pp 2199–2203. https://doi.org/10.1109/iccsp.2016.7754573

  41. 41.

    Song W, Deng B, Zhang H, Xiao Q, Peng S (2016) An adaptive real-time video defogging method based on context-sensitiveness. In: 2016 IEEE international conference on real-time computing and robotics (RCAR), 6–10 June 2016, pp 406–410. https://doi.org/10.1109/rcar.2016.7784063

  42. 42.

    Guo JM, Syue JY, Radzicki VR, Lee H (2017) An efficient fusion-based defogging. IEEE Trans Image Process 26(9):4217–4228. https://doi.org/10.1109/tip.2017.2706526

    MathSciNet  Article  MATH  Google Scholar 

  43. 43.

    Li Y, You S, Brown MS, Tan RT (2017) Haze visibility enhancement: a survey and quantitative benchmarking. Comput Vis Image Underst 165:1–16. https://doi.org/10.1016/j.cviu.2017.09.003

    Article  Google Scholar 

  44. 44.

    Ancuti C, Ancuti CO, Vleeschouwer CD (2016) D-HAZY: a dataset to evaluate quantitatively dehazing algorithms. In: 2016 IEEE international conference on image processing (ICIP), 25–28 Sept 2016, pp 2226–2230. https://doi.org/10.1109/icip.2016.7532754

  45. 45.

    Goswami S, Kumar J, Goswami J (2015) A hybrid approach for visibility enhancement in foggy image. In: 2015 2nd International conference on computing for sustainable global development (INDIACom), 11–13 March 2015, pp 175–180

  46. 46.

    Zhang W, Liang J, Ju H, Ren L, Qu E, Wu Z (2016) A robust haze-removal scheme in polarimetric dehazing imaging based on automatic identification of sky region. Opt Laser Technol 86:145–151. https://doi.org/10.1016/j.optlastec.2016.07.015

    Article  Google Scholar 

  47. 47.

    Pham TY, Ma HM, Yeo GT (2017) Application of Fuzzy Delphi TOPSIS to locate logistics centers in Vietnam: the Logisticians’ perspective. Asian J Shipping Logist 33(4):211–219

    Google Scholar 

  48. 48.

    Sharifabadi AM, Sadrabadi AN, Bezegabadi FD, Peirow S, Taki E (2015) Presenting a model for evaluation and selecting suppliers using interpretive structure modeling (ISM). Int J Acad Res 27(2):109–120

    Google Scholar 

  49. 49.

    Kamarulzaman N, Jomhari N, Raus NM, Yusoff MZM (2015) Applying the fuzzy delphi method to analyze the user requirement for user centred design process in order to create learning applications. Indian J Sci Technol 8(32):1–17

    Google Scholar 

  50. 50.

    Rahimianzarif E, Moradi M (2018) Designing integrated management criteria of creative ideation based on fuzzy delphi analytical hierarchy process. Int J Fuzzy Syst 20(3):877–900

    Google Scholar 

  51. 51.

    Manakandan SK, Rosnah I, Mohd JR, Priya R (2017) Pesticide applicators questionnaire content validation: a fuzzy delphi method. Med J Malays 72(4):228–235

    Google Scholar 

  52. 52.

    Zhao H, Li N (2016) Optimal siting of charging stations for electric vehicles based on fuzzy Delphi and hybrid multi-criteria decision making approaches from an extended sustainability perspective. Energies 9(4):270

    Google Scholar 

  53. 53.

    Hsu Y-L, Lee C-H, Kreng VB (2010) The application of Fuzzy Delphi method and Fuzzy AHP in lubricant regenerative technology selection. Expert Syst Appl 37(1):419–425

    Google Scholar 

  54. 54.

    Lee S, Seo K-K (2016) A hybrid multi-criteria decision-making model for a cloud service selection problem using BSC, fuzzy Delphi method and fuzzy AHP. Wireless Pers Commun 86(1):57–75

    Google Scholar 

  55. 55.

    Tahriri F, Mousavi M, Haghighi SH, Dawal SZM (2014) The application of fuzzy Delphi and fuzzy inference system in supplier ranking and selection. J Ind Eng Int 10(3):66

    Google Scholar 

  56. 56.

    Alsalem M et al (2018) Systematic review of an automated multiclass detection and classification system for acute Leukaemia in terms of evaluation and benchmarking, open challenges, issues and methodological aspects. J Med Syst 42(11):204

    Google Scholar 

  57. 57.

    Albahri A, Zaidan A, Albahri O, Zaidan B, Alsalem M (2018) Real-time fault-tolerant mhealth system: comprehensive review of healthcare services, opens issues, challenges and methodological aspects. J Med Syst 42(8):137

    Google Scholar 

  58. 58.

    Yas QM, Zaidan A, Zaidan B, Rahmatullah B, Karim HA (2017) Comprehensive insights into evaluation and benchmarking of real-time skin detectors: review, open issues & challenges, and recommended solutions. Measurement 114:243–260

    Google Scholar 

  59. 59.

    Whaiduzzaman M, Gani A, Anuar NB, Shiraz M, Haque MN, Haque IT (2014) Cloud service selection using multicriteria decision analysis. Sci World J 2014:1–10

  60. 60.

    Kalid N et al (2018) Based on real time remote health monitoring systems: a new approach for prioritization “large scales data” patients with chronic heart diseases using body sensors and communication technology. J Med Syst 42(4):69

    Google Scholar 

  61. 61.

    Albahri A et al (2019) Based multiple heterogeneous wearable sensors: a smart real-time health monitoring structured for hospitals distributor. IEEE Access 7:37269–37323

    Google Scholar 

  62. 62.

    Albahri O et al (2019) Fault-tolerant mHealth framework in the context of IoT-based real-time wearable health data sensors. IEEE Access 7:50052–50080

    Google Scholar 

  63. 63.

    Zaidan A et al (2018) A review on smartphone skin cancer diagnosis apps in evaluation and benchmarking: coherent taxonomy, open issues and recommendation pathway solution. Health Technol 8:1–16

    Google Scholar 

  64. 64.

    Petrovic-Lazarevic S, Abraham A (2004) Hybrid fuzzy-linear programming approach for multi criteria decision making problems. arXiv preprint cs/0405019

  65. 65.

    Zaidan A, Zaidan B, Al-Haiqi A, Kiah MLM, Hussain M, Abdulnabi M (2015) Evaluation and selection of open-source EMR software packages based on integrated AHP and TOPSIS. J Biomed Inform 53:390–404

    Google Scholar 

  66. 66.

    Zaidan A, Zaidan B, Hussain M, Haiqi A, Kiah MM, Abdulnabi M (2015) Multi-criteria analysis for OS-EMR software selection problem: a comparative study. Decis Support Syst 78:15–27

    Google Scholar 

  67. 67.

    Abdullateef BN, Elias NF, Mohamed H, Zaidan A, Zaidan B (2016) An evaluation and selection problems of OSS-LMS packages. SpringerPlus 5(1):248

    Google Scholar 

  68. 68.

    Yas QM, Zadain A, Zaidan B, Lakulu M, Rahmatullah B (2017) Towards on develop a framework for the evaluation and benchmarking of skin detectors based on artificial intelligent models using multi-criteria decision-making techniques. Int J Pattern Recognit Artif Intell 31(03):1759002

    Google Scholar 

  69. 69.

    Zaidan B, Zaidan A, Karim HA, Ahmad N (2017) A new digital watermarking evaluation and benchmarking methodology using an external group of evaluators and multi-criteria analysis based on ‘large-scale data’. Softw Pract Exp 47(10):1365–1392

    Google Scholar 

  70. 70.

    Malczewski J (1999) GIS and multicriteria decision analysis. Wiley, New York

    Google Scholar 

  71. 71.

    Zaidan B, Zaidan A (2017) Software and hardware FPGA-based digital watermarking and steganography approaches: toward new methodology for evaluation and benchmarking using multi-criteria decision-making techniques. J Circuits Syst Comput 26(07):1750116

    Google Scholar 

  72. 72.

    Zaidan B, Zaidan A, Abdul Karim H, Ahmad N (2017) A new approach based on multi-dimensional evaluation and benchmarking for data hiding techniques. Int J Inf Technol Decis Mak 16:1–42

  73. 73.

    Jumaah F, Zaidan A, Zaidan B, Bahbibi R, Qahtan M, Sali A (2018) Technique for order performance by similarity to ideal solution for solving complex situations in multi-criteria optimization of the tracking channels of GPS baseband telecommunication receivers. Telecommun Syst 68(3):425–443

    Google Scholar 

  74. 74.

    Rahmatullah B, Zaidan A, Mohamed F, Sali A (2017) Multi-complex attributes analysis for optimum GPS baseband receiver tracking channels selection. In: 2017 4th international conference on control, decision and information technologies (CoDIT), 2017. IEEE, pp 1084–1088

  75. 75.

    Salman OH, Zaidan A, Zaidan B, Naserkalid A, Hashim M (2017) Novel methodology for triage and prioritizing using “big data” patients with chronic heart diseases through telemedicine environmental. Int J Inf Technol Decis Mak 16(05):1211–1245

    Google Scholar 

  76. 76.

    Zaidan B, Zaidan A (2018) Comparative study on the evaluation and benchmarking information hiding approaches based multi-measurement analysis using TOPSIS method with different normalisation, separation and context techniques. Measurement 117:277–294

    Google Scholar 

  77. 77.

    Oliveira M, Fontes DB, Pereira T (2014) Multicriteria decision making: a case study in the automobile industry. Ann Manag Sci 3(1):109

    Google Scholar 

  78. 78.

    AlSattar H et al (2018) MOGSABAT: a metaheuristic hybrid algorithm for solving multi-objective optimisation problems. Neural Comput Appl 32:1–15

  79. 79.

    Enaizan O et al (2018) Electronic medical record systems: decision support examination framework for individual, security and privacy concerns using multi-perspective analysis. Health Technol 1–28

  80. 80.

    Salih MM, Zaidan B, Zaidan A, Ahmed MA (2019) Survey on fuzzy TOPSIS state-of-the-art between 2007 and 2017. Comput Oper Res 104:207–227

    MathSciNet  MATH  Google Scholar 

  81. 81.

    Kalid N, Zaidan A, Zaidan B, Salman OH, Hashim M, Muzammil H (2018) Based real time remote health monitoring systems: a review on patients prioritization and related “big data” using body sensors information and communication technology. J Med Syst 42(2):30

    Google Scholar 

  82. 82.

    Jumaah F, Zadain A, Zaidan B, Hamzah A, Bahbibi R (2018) Decision-making solution based multi-measurement design parameter for optimization of GPS receiver tracking channels in static and dynamic real-time positioning multipath environment. Measurement 118:83–95

    Google Scholar 

  83. 83.

    Albahri O, Zaidan A, Zaidan B, Hashim M, Albahri A, Alsalem M (2018) Real-time remote health-monitoring Systems in a Medical Centre: a review of the provision of healthcare services-based body sensor information, open challenges and methodological aspects. J Med Syst 42(9):164

    Google Scholar 

  84. 84.

    Zaidan A, Zaidan B, Alsalem M, Albahri O, Albahri A, Qahtan M (2019) Multi-agent learning neural network and Bayesian model for real-time IoT skin detectors: a new evaluation and benchmarking methodology. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04325-3

  85. 85.

    Albahri O et al (2018) Systematic review of real-time remote health monitoring system in triage and priority-based sensor technology: taxonomy, open challenges, motivation and recommendations. J Med Syst 42(5):80

    Google Scholar 

  86. 86.

    Lim C, Tan K, Zaidan A, Zaidan B (2020) A proposed methodology of bringing past life in digital cultural heritage through crowd simulation: a case study in George Town, Malaysia. Multimed Tools Appl 79(5):3387–3423

    Google Scholar 

  87. 87.

    Napi N, Zaidan A, Zaidan B, Albahri O, Alsalem M, Albahri A (2019) Medical emergency triage and patient prioritisation in a telemedicine environment: a systematic review. Health Technol 9:1–22

  88. 88.

    Jadhav A, Sonar R (2009) Analytic hierarchy process (AHP), weighted scoring method (WSM), and hybrid knowledge based system (HKBS) for software selection: a comparative study. In: 2009 Second international conference on emerging trends in engineering and technology. IEEE, pp 991–997

  89. 89.

    Khatari M, Zaidan A, Zaidan B, Albahri O, Alsalem M (2019) Multi-criteria evaluation and benchmarking for active queue management methods: open issues challenges and recommended pathway solutions. Int J Inf Technol Decis Mak 18(4):1187–1242

    Google Scholar 

  90. 90.

    Almahdi E, Zaidan A, Zaidan B, Alsalem M, Albahri O, Albahri A (2019) Mobile patient monitoring systems from a benchmarking aspect: challenges, open issues and recommended solutions. J Med Syst 43(7):207

    Google Scholar 

  91. 91.

    Almahdi E, Zaidan A, Zaidan B, Alsalem M, Albahri O, Albahri A (2019) Mobile-based patient monitoring systems: a prioritisation framework using multi-criteria decision-making techniques. J Med Syst 43(7):219

    Google Scholar 

  92. 92.

    Mohammed K et al (2019) Real-time remote-health monitoring systems: a review on patients prioritisation for multiple-chronic diseases, taxonomy analysis, concerns and solution procedure. J Med Syst 43(7):223

    Google Scholar 

  93. 93.

    Alaa M et al (2019) Assessment and ranking framework for the English skills of pre-service teachers based on fuzzy Delphi and TOPSIS methods. IEEE Access 7:126201–126223

    Google Scholar 

  94. 94.

    Ibrahim N et al (2019) Multi-criteria evaluation and benchmarking for Young learners’ english language mobile applications in terms of LSRW skills. IEEE Access 7:146620–146651

    Google Scholar 

  95. 95.

    Talal M et al (2019) Comprehensive review and analysis of anti-malware apps for smartphones. Telecommun Syst 72(2):285–337

    Google Scholar 

  96. 96.

    Nedher A-S, Hassan S, Katuk N (2014) On multi attribute decision making methods: prioritizing information security controls. J Appl Sci 14(16):1865–1870

    Google Scholar 

  97. 97.

    Mohammed K et al (2020) Novel technique for reorganisation of opinion order to interval levels for solving several instances representing prioritisation in patients with multiple chronic diseases. Comput Methods Progr Biomed 185:105151

    Google Scholar 

  98. 98.

    Hongjiu L, Yanrong H (2015) An evaluating method with combined assigning-weight based on maximizing variance. Sci Program 2015:3

    Google Scholar 

  99. 99.

    Zou Z-H, Yi Y, Sun J-N (2006) Entropy method for determination of weight of evaluating indicators in fuzzy synthetic evaluation for water quality assessment. J Environ Sci 18(5):1020–1023

    Google Scholar 

  100. 100.

    Opricovic S, Tzeng G-H (2004) Compromise solution by MCDM methods: a comparative analysis of VIKOR and TOPSIS. Eur J Oper Res 156(2):445–455

    MATH  Google Scholar 

  101. 101.

    Opricovic S, Tzeng G-H (2007) Extended VIKOR method in comparison with outranking methods. Eur J Oper Res 178(2):514–529

    MATH  Google Scholar 

  102. 102.

    Mahjouri M, Ishak MB, Torabian A, Manaf LA, Halimoon N, Ghoddusi J (2017) Optimal selection of iron and steel wastewater treatment technology using integrated multi-criteria decision-making techniques and fuzzy logic. Process Saf Environ Prot 107:54–68

    Google Scholar 

  103. 103.

    Shemshadi A, Shirazi H, Toreihi M, Tarokh MJ (2011) A fuzzy VIKOR method for supplier selection based on entropy measure for objective weighting. Expert Syst Appl 38(10):12160–12167

    Google Scholar 

  104. 104.

    Malekian A, Azarnivand A (2016) Application of integrated Shannon’s entropy and VIKOR techniques in prioritization of flood risk in the Shemshak watershed, Iran. Water Resour Manag 30(1):409–425

    Google Scholar 

  105. 105.

    Bhuyan R, Routara B (2016) Optimization the machining parameters by using VIKOR and entropy weight method during EDM process of Al-18% SiCp metal matrix composite. Decis Sci Lett 5(2):269–282

    Google Scholar 

  106. 106.

    Mohsen O, Fereshteh N (2017) An extended VIKOR method based on entropy measure for the failure modes risk assessment: a case study of the geothermal power plant (GPP). Saf Sci 92:160–172

    Google Scholar 

  107. 107.

    Mardani A, Zavadskas EK, Govindan K, Amat Senin A, Jusoh A (2016) VIKOR technique: a systematic review of the state of the art literature on methodologies and applications. Sustainability 8(1):37

    Google Scholar 

  108. 108.

    Cheng C-H, Lin Y (2002) Evaluating the best main battle tank using fuzzy decision theory with linguistic criteria evaluation. Eur J Oper Res 142(1):174–186

    MATH  Google Scholar 

  109. 109.

    Chu H-C, Hwang G-J (2008) A Delphi-based approach to developing expert systems with the cooperation of multiple experts. Expert Syst Appl 34(4):2826–2840

    Google Scholar 

  110. 110.

    Murry JW Jr, Hammons JO (1995) Delphi: a versatile methodology for conducting qualitative research. Rev Higher Educ 18(4):423–436

    Google Scholar 

  111. 111.

    Bekri RM, Ruhizan M, Norazah M, Nur YFA, Ashikin HT (2013) Development of Malaysia skills certificate E-portfolio: a conceptual framework. Proc Soc Behav Sci 103:323–329

    Google Scholar 

  112. 112.

    Bodjanova S (2006) Median alpha-levels of a fuzzy number. Fuzzy Sets Syst 157(7):879–891

    MathSciNet  MATH  Google Scholar 

  113. 113.

    Tang C-W, Wu C-T (2010) Obtaining a picture of undergraduate education quality: a voice from inside the university. High Educ 60(3):269–286

    Google Scholar 

  114. 114.

    Choi LK, You J, Bovik AC (2015) Referenceless prediction of perceptual fog density and perceptual image defogging. IEEE Trans Image Process 24(11):3888–3901. https://doi.org/10.1109/TIP.2015.2456502

    MathSciNet  Article  MATH  Google Scholar 

  115. 115.

    Cai B, Xu X, Jia K, Qing C, Tao D (2016) DehazeNet: an end-to-end system for single image haze removal. IEEE Trans Image Process 25(11):5187–5198. https://doi.org/10.1109/TIP.2016.2598681

    MathSciNet  Article  MATH  Google Scholar 

  116. 116.

    Ren W, Liu S, Zhang H, Pan J, Cao X, Yang M-H (2016) Single image dehazing via multi-scale convolutional neural networks. In: European conference on computer vision. Springer, pp 154–169

  117. 117.

    Salazar-Colores S, Cruz-Aceves I, Ramos-Arreguin J-M (2018) Single image dehazing using a multilayer perceptron. J Electron Imaging 27(4):043022

    Google Scholar 

  118. 118.

    Zhu Q, Mai J, Shao L (2015) A fast single image haze removal algorithm using color attenuation prior. IEEE Trans Image Process 24(11):3522–3533

    MathSciNet  MATH  Google Scholar 

  119. 119.

    Cho Y, Jeong J, Kim A (2018) Model-assisted multiband fusion for single image enhancement and applications to robot vision. IEEE Robot Autom Lett 3(4):2822–2829

    Google Scholar 

  120. 120.

    He J, Zhang C, Yang R, Zhu K (2016) Convex optimization for fast image dehazing. In: 2016 IEEE international conference on image processing (ICIP), 25–28 Sept 2016, pp 2246–2250. https://doi.org/10.1109/icip.2016.7532758

  121. 121.

    Meng G, Wang Y, Duan J, Xiang S, Pan C (2013) Efficient image dehazing with boundary constraint and contextual regularization. In: Proceedings of the IEEE international conference on computer vision, pp 617–624

  122. 122.

    Liu X, Zhang H, Cheung Y-M, You X, Tang YY (2017) Efficient single image dehazing and denoising: an efficient multi-scale correlated wavelet approach. Comput Vis Image Underst 162:23–33. https://doi.org/10.1016/j.cviu.2017.08.002

    Article  Google Scholar 

  123. 123.

    Berman D, Avidan S (2016) Non-local image dehazing. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1674–1682

  124. 124.

    Lotfi FH, Fallahnejad R (2010) Imprecise Shannon’s entropy and multi attribute decision making. Entropy 12(1):53–62

    MATH  Google Scholar 

  125. 125.

    Wu J, Sun J, Liang L, Zha Y (2011) Determination of weights for ultimate cross efficiency using Shannon entropy. Expert Syst Appl 38(5):5162–5165

    Google Scholar 

  126. 126.

    Alsalem M et al (2019) Multiclass benchmarking framework for automated acute Leukaemia detection and classification based on BWM and group-VIKOR. J Med Syst 43(7):212

    Google Scholar 

  127. 127.

    Triantaphyllou E, Baig K (2005) The impact of aggregating benefit and cost criteria in four MCDA methods. IEEE Trans Eng Manag 52(2):213–226

    Google Scholar 

  128. 128.

    Mullen PM (2003) Delphi: myths and reality. J Health Organ Manag 17(1):37–52

    Google Scholar 

  129. 129.

    Bueno S, Salmeron JL (2008) Fuzzy modeling enterprise resource planning tool selection. Comput Stand Interfaces 30(3):137–147

    Google Scholar 

  130. 130.

    Manoliadis O, Tsolas I, Nakou A (2006) Sustainable construction and drivers of change in Greece: a Delphi study. Constr Manag Econ 24(2):113–120

    Google Scholar 

  131. 131.

    Mohamad SNA, Embi MA, Nordin N (2015) Determining e-Portfolio elements in learning process using fuzzy Delphi analysis. Int Educ Stud 8(9):171–176

    Google Scholar 

  132. 132.

    Chang P-L, Hsu C-W, Chang P-C (2011) Fuzzy Delphi method for evaluating hydrogen production technologies. Int J Hydrog Energy 36(21):14172–14179

    Google Scholar 

  133. 133.

    Qader M, Zaidan B, Zaidan A, Ali S, Kamaluddin M, Radzi W (2017) A methodology for football players selection problem based on multi-measurements criteria analysis. Measurement 111:38–50

    Google Scholar 

  134. 134.

    Wu D, Zhu Q, Wang J, Xie Y, Wang L (2014) Image haze removal: status, challenges and prospects. In: 2014 4th IEEE international conference on information science and technology, 26–28 April 2014, pp 492–497. https://doi.org/10.1109/icist.2014.6920524

  135. 135.

    Duarte A, Codevilla F, Gaya JDO, Botelho SSC (2016) A dataset to evaluate underwater image restoration methods. In: OCEANS 2016—Shanghai, 10–13 April 2016, pp 1–6. https://doi.org/10.1109/oceansap.2016.7485524

  136. 136.

    Li Y, Wang K, Xu N, Li Y (2017) Quantitative evaluation for dehazing algorithms on synthetic outdoor hazy dataset. In: 2017 IEEE visual communications and image processing (VCIP), 10–13 Dec 2017, pp 1–4. https://doi.org/10.1109/vcip.2017.8305081

  137. 137.

    Yadav G, Maheshwari S, Agarwal A (2014) Fog removal techniques from images: a comparative review and future directions. In: 2014 international conference on signal propagation and computer technology (ICSPCT 2014), 12–13 July 2014, pp 44–52. https://doi.org/10.1109/icspct.2014.6884973

  138. 138.

    Pal T, Bhowmik MK, Bhattacharjee D, Ghosh AK (2016) Visibility enhancement techniques for fog degraded images: a comparative analysis with performance evaluation. In: 2016 IEEE region 10 conference (TENCON), 22–25 Nov 2016, pp 2583–2588. https://doi.org/10.1109/tencon.2016.7848504

  139. 139.

    Roy SD, Bhowmik MK, Saha SS (2017) Qualitative evaluation of visibility enhancement techniques on SAMEER-TU database for security and surveillance. In: 2017 8th international conference on computing, communication and networking technologies (ICCCNT), 3–5 July 2017, pp 1–7. https://doi.org/10.1109/icccnt.2017.8204002

  140. 140.

    Wang W, Chang F, Ji T, Wu X (2018) A fast single-image dehazing method based on a physical model and gray projection. IEEE Access 6:5641–5653. https://doi.org/10.1109/ACCESS.2018.2794340

    Article  Google Scholar 

  141. 141.

    Chen BH, Huang SC, Cheng FC (2016) A high-efficiency and high-speed gain intervention refinement filter for haze removal. J Disp Technol 12(7):753–759. https://doi.org/10.1109/JDT.2016.2518646

    Article  Google Scholar 

  142. 142.

    Zhang T, Hu HM, Li B (2018) A naturalness preserved fast dehazing algorithm using HSV color space. IEEE Access. https://doi.org/10.1109/access.2018.2806372

    Article  Google Scholar 

  143. 143.

    Khodary AG, Aly HA, IEEE (2014) A new image-sequence haze removal system based on DM6446 Davinci processor. In: 2014 IEEE global conference on signal and information processing, pp 703–706

  144. 144.

    El-Hashash MM, Aly HA, Mahmoud TA, Swelam W (2015) A video haze removal system on heterogeneous cores. In: 2015 IEEE global conference on signal and information processing (GlobalSIP), pp 1255–1259. https://doi.org/10.1109/globalsip.2015.7418399

  145. 145.

    Changli L, Tanghuai F, Xiao M, Zhen Z, Hongxin W, Lin C (2017) An improved image defogging method based on dark channel prior. In: 2017 2nd international conference on image, vision and computing (ICIVC), 2–4 June 2017, pp 414–417. https://doi.org/10.1109/icivc.2017.7984589

  146. 146.

    Guo F, Cai Z, Xie B, Tang J (2010) Automatic image haze removal based on luminance component. In: 2010 6th International conference on wireless communications networking and mobile computing (WiCOM), 23–25 Sept 2010, pp 1–4. https://doi.org/10.1109/wicom.2010.5600632

  147. 147.

    Mai J, Zhu Q, Wu D, Xie Y, Wang (2014) Back propagation neural network dehazing. In: 2014 IEEE international conference on robotics and biomimetics (ROBIO 2014), 5–10 Dec, pp 1433–1438. https://doi.org/10.1109/robio.2014.7090535

  148. 148.

    Nair D, Sankaran P (2018) Color image dehazing using surround filter and dark channel prior. J Vis Commun Image Represent 50:9–15

    Google Scholar 

  149. 149.

    Yadav G, Maheshwari S, Agarwal A (2014) Contrast limited adaptive histogram equalization based enhancement for real time video system. In: 2014 International conference on advances in computing, communications and informatics (ICACCI), 24–27 Sept 2014, pp 2392–2397. https://doi.org/10.1109/icacci.2014.6968381

  150. 150.

    Roy K, Kumar S, Banerjee S, Sarkar TS, Chaudhuri SS (2017) Dehazing technique for natural scene image based on color analysis and restoration with road edge detection. In: 2017 1st International conference on electronics, materials engineering and nano-technology (IEMENTech), 28–29 April 2017, pp 1–6. https://doi.org/10.1109/iementech.2017.8076989

  151. 151.

    Ancuti CO, Ancuti C, Bekaert P (2010) Effective single image dehazing by fusion. In: 2010 IEEE international conference on image processing, 26–29 Sept 2010, pp 3541–3544. https://doi.org/10.1109/icip.2010.5651263

  152. 152.

    Liao B, Yin P, Xiao C (2018) Efficient image dehazing using boundary conditions and local contrast. Comput Graph 70:242–250. https://doi.org/10.1016/j.cag.2017.07.016

    Article  Google Scholar 

  153. 153.

    Negru M, Nedevschi S, Peter RI (2015) Exponential contrast restoration in fog conditions for driving assistance. IEEE Trans Intell Transp Syst 16(4):2257–2268. https://doi.org/10.1109/TITS.2015.2405013

    Article  Google Scholar 

  154. 154.

    Negru M, Nedevschi S, Peter RI (2014) Exponential image enhancement in daytime fog conditions. In: 17th International IEEE conference on intelligent transportation systems (ITSC), 8–11 Oct 2014, pp 1675–1681. https://doi.org/10.1109/itsc.2014.6957934

  155. 155.

    Kumari A, Kodati H, Sahoo SK (2015) Fast and efficient contrast enhancement for real time video dehazing and defogging. In: 2015 IEEE workshop on computational intelligence: theories, applications and future directions (WCI), 14–17 Dec 2015, 14–17 Dec 2015, pp 1–5. https://doi.org/10.1109/wci.2015.7495527

  156. 156.

    Qian X, Han L (2014) Fast image dehazing algorithm based on multiple filters. In: 2014 10th international conference on natural computation (ICNC), 19–21 Aug 2014, pp 937–0941. https://doi.org/10.1109/icnc.2014.6975965

  157. 157.

    Wang W, Yuan X, Wu X, Liu Y (2017) Fast image dehazing method based on linear transformation. IEEE Trans Multimed 19(6):1142–1155. https://doi.org/10.1109/TMM.2017.2652069

    Article  Google Scholar 

  158. 158.

    Zhang X, Bu Z, Chen H, Liu M (2015) Fast image dehazing using joint Local Linear sure-based filter and image fusion. In 2015 5th international conference on information science and technology (ICIST), 24–26 Apr 2015, pp 192–197. https://doi.org/10.1109/icist.2015.7288966

  159. 159.

    Zhu X, Li Y, Qiao Y (2015) Fast single image dehazing through Edge-Guided Interpolated Filter. In: 2015 14th IAPR international conference on machine vision applications (MVA), 18–22 May 2015, pp 443–446. https://doi.org/10.1109/mva.2015.7153106

  160. 160.

    Zhang B, Zhao J (2017) Hardware implementation for real-time haze removal. IEEE Trans Very Large Scale Integr VLSI Syst 25(3):1188–1192. https://doi.org/10.1109/tvlsi.2016.2622404

    Article  Google Scholar 

  161. 161.

    Zhao X, Ding W, Liu C, Li H (2018) Haze removal for unmanned aerial vehicle aerial video based on spatial-temporal coherence optimisation. IET Image Proc 12(1):88–97. https://doi.org/10.1049/iet-ipr.2017.0060

    Article  Google Scholar 

  162. 162.

    Liu S et al (2017) Image de-hazing from the perspective of noise filtering. Comput Electr Eng 62:345–359. https://doi.org/10.1016/j.compeleceng.2016.11.021

    Article  Google Scholar 

  163. 163.

    Huang C, Yang D, Zhang R, Wang L, Zhou L (2017) Improved algorithm for image haze removal based on dark channel priority. Comput Electr Eng. https://doi.org/10.1016/j.compeleceng.2017.09.018

    Article  Google Scholar 

  164. 164.

    Xie B, Guo F, Cai Z (2010) Improved Single Image Dehazing Using Dark Channel Prior and Multi-scale Retinex. In 2010 international conference on intelligent system design and engineering application, 13–14 Oct 2010, vol 1, pp 848–851. https://doi.org/10.1109/isdea.2010.141

  165. 165.

    Zhi W, Watabe D, Jianting C (2016) Improving visibility of a fast dehazing method. In 2016 world automation congress (WAC), July 31 2016–Aug 4 2016, pp 1-6. https://doi.org/10.1109/wac.2016.7582960

  166. 166.

    Liu H, Huang D, Hou S, Yue R (2017) Large size single image fast defogging and the real time video defogging FPGA architecture. Neurocomputing 269:97–107. https://doi.org/10.1016/j.neucom.2016.09.139

    Article  Google Scholar 

  167. 167.

    Hautiere N, Tarel JP, Aubert D (2010) Mitigation of visibility loss for advanced camera-based driver assistance. IEEE Trans Intell Transp Syst 11(2):474–484. https://doi.org/10.1109/TITS.2010.2046165

    Article  Google Scholar 

  168. 168.

    Kumari A, Sahoo SK (2015) Real time visibility enhancement for single image haze removal. Procedia Comput Sci 54:501–507. https://doi.org/10.1016/j.procs.2015.06.057

    Article  Google Scholar 

  169. 169.

    Zhang J, Ding Y, Yang Y, Sun J (2016) Real-time defog model based on visible and near-infrared information. In: 2016 IEEE international conference on multimedia & expo workshops (ICMEW), 11–15 July 2016, pp 1–6. https://doi.org/10.1109/icmew.2016.7574749

  170. 170.

    Ji X, Feng Y, Liu G, Dai M, Yin C (2010) Real-time defogging processing of aerial images. In: 2010 6th international conference on wireless communications networking and mobile computing (WiCOM), 23–25 Sept 2010, pp 1–4. https://doi.org/10.1109/wicom.2010.5600245

  171. 171.

    Alajarmeh A, Salam RA, Abdulrahim K, Marhusin MF, Zaidan AA, Zaidan BB (2018) Real-time framework for image dehazing based on linear transmission and constant-time airlight estimation. Inf Sci 436–437:108–130. https://doi.org/10.1016/j.ins.2018.01.009

    MathSciNet  Article  Google Scholar 

  172. 172.

    Yu T, Riaz I, Piao J, Shin H (2015) Real-time single image dehazing using block-to-pixel interpolation and adaptive dark channel prior. IET Image Proc 9(9):725–734. https://doi.org/10.1049/iet-ipr.2015.0087

    Article  Google Scholar 

  173. 173.

    Liu X, Zhang H, Tang YY, Du JX (2016) Scene-adaptive single image dehazing via opening dark channel model. IET Image Proc 10(11):877–884. https://doi.org/10.1049/iet-ipr.2016.0138

    Article  Google Scholar 

  174. 174.

    Liu X, Zeng F, Huang Z, Ji Y (2013) Single color image dehazing based on digital total variation filter with color transfer. In: 2013 IEEE international conference on image processing, 15–18 Sept 2013, pp 909–913. https://doi.org/10.1109/icip.2013.6738188

  175. 175.

    Huang D, Chen K, Lu, J, Wang W (2017) Single image dehazing based on deep neural network. In: 2017 international conference on computer network, electronic and automation (ICCNEA), 23–25 Sept 2017, pp 294–299. https://doi.org/10.1109/iccnea.2017.107

  176. 176.

    Ancuti CO, Ancuti C (2013) Single image Dehazing by multi-scale fusion. IEEE Trans Image Process 22(8):3271–3282. https://doi.org/10.1109/TIP.2013.2262284

    Article  Google Scholar 

  177. 177.

    Bui TM, Kim W (2018) Single image dehazing using color ellipsoid prior. IEEE Trans Image Process 27(2):999–1009. https://doi.org/10.1109/TIP.2017.2771158

    MathSciNet  Article  MATH  Google Scholar 

  178. 178.

    Riaz I, Yu T, Rehman Y, Shin H (2016) Single image dehazing via reliability guided fusion. J Vis Commun Image Represent 40(Part A):85–97. https://doi.org/10.1016/j.jvcir.2016.06.011

    Article  Google Scholar 

  179. 179.

    Zhao H, Xiao C, Yu J, Xu X (2015) Single image fog removal based on local extrema. IEEE/CAA J Autom Sin 2(2):158–165. https://doi.org/10.1109/JAS.2015.7081655

    MathSciNet  Article  Google Scholar 

  180. 180.

    Tripathi AK, Mukhopadhyay S (2012) Single image fog removal using anisotropic diffusion. IET Image Proc 6(7):966–975. https://doi.org/10.1049/iet-ipr.2011.0472

    MathSciNet  Article  Google Scholar 

  181. 181.

    Gao Z, Bai Y (2016) Single image haze removal algorithm using pixel-based airlight constraints. In: 2016 22nd international conference on automation and computing (ICAC), 7–8 Sept 2016, pp 267–272. https://doi.org/10.1109/iconac.2016.7604930

  182. 182.

    Serikawa S, Lu H (2014) Underwater image dehazing using joint trilateral filter. Comput Electr Eng 40(1):41–50. https://doi.org/10.1016/j.compeleceng.2013.10.016

    Article  Google Scholar 

  183. 183.

    Xie B, Guo F, Cai ZX (2012) Universal strategy for surveillance video defogging. Opti Eng 51(10), Art no. 101703. https://doi.org/10.1117/1.oe.51.10.101703

  184. 184.

    Shiau YH, Kuo YT, Chen PY, Hsu FY (2017) VLSI design of an efficient flicker-free video defogging method for real-time applications. IEEE Trans Circuits Syst Video Technol PP(99):1. https://doi.org/10.1109/tcsvt.2017.2777140

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to A. A. Zaidan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.



See Table 11.

Table 11 Classification and frequency of usage for image dehazing criteria

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Abdulkareem, K.H., Arbaiy, N., Zaidan, A.A. et al. A new standardisation and selection framework for real-time image dehazing algorithms from multi-foggy scenes based on fuzzy Delphi and hybrid multi-criteria decision analysis methods. Neural Comput & Applic 33, 1029–1054 (2021). https://doi.org/10.1007/s00521-020-05020-4

Download citation


  • Real-time image dehazing algorithms
  • Standardisation
  • Selection
  • Fuzzy Delphi method
  • Entropy