Design of Experiments

Abstract

The data from which performance models are identified may originate either from planned experiments or from non-intrusive (or observational) data gathered while the system is in normal operation. A large body of well accepted practices is available for the former which falls under the general terminology of “Design of Experiments” (DOE). This is the process of defining the structural framework, i.e., prescribing the exact manner in which samples for testing need to be selected, and the conditions and sequence under which the testing needs to be performed. This would provide the “richness” in the data set necessary for statistically sound performance models to be identified between the response variable and the several categorical factors. Experimental design methods, which allow extending hypothesis testing to multiple variables as well as identifying sound performance models, are presented. Selected experimental design methods are discussed such as randomized block, Latin Squares and 2^k factorial designs. The parallel between model building in a DOE framework and linear multiple regression is illustrated. Finally, this chapter addresses response surface methods (RSM) which allow accelerating the search towards optimizing a process or towards finding the conditions under which a desirable behavior of a product is optimized. RSM is a sequential approach where one starts with test conditions in a plausible area of the search space, analyzes test results to determine the optimal direction to move, performs a second set of test conditions, and so on till the required optimum is reached.

Problems

Pr. 6.1

Consider Example 6.2.2 where the performance of four machines was analyzed in terms of machining time with operator dexterity being a factor to be blocked. How to identify an additive linear model was also illustrated. Figure 6.2a suggests that interaction effects may be important. You will re-analyze the data to determine whether interaction terms are statistically significant or not.

Pr. 6.2

Full-factorial design for evaluating three different missile systems

A full-factorial experiment is conducted to determine which of 3 different missile systems is preferable. The propellant burning rate for 24 static firings was measured using four different propellant types. The experiment performed duplicate observations (replicate = 2) of burning rates (in minutes) at each combination of the treatments. The data, after coding, is given in Table 6.26.

Table 6.26 Burning rates in minutes for the (3 × 4) case with two replicates. (Problem 6.2)

The following hypotheses tests are to be studied:

• (a) There is no difference in the mean propellant burning rates when different missile systems are used,

• (b) there is no difference in the mean propellant burning rates of the 4 propellant types,

• (c) there is no interaction between the different missile systems and the different propellant types,

Pr. 6.3

Random effects model for worker productivity

A full-factorial experiment was conducted to study the effect of indoor environment condition (depending on such factors as dry bulb temperature, relative humidity…) on the productivity of workers manufacturing widgets. Four groups of workers were selected distinguished by such traits as age, gender,… called G1, G2, G3 and G4. The number of widgets produced over a day by two members of each group under three different environmental conditions (E1, E2 and E3) was recorded. These results are assembled in Table 6.27.

Table 6.27 Showing the number of widgets produced by day using a replicate r = 2. (Problem 6.3)

Using 0.05 significance level, test the hypothesis that:

• (a) different environmental conditions have no effect on number of widgets produced,

• (b) different worker groups have no effect on number of widgets produced,

• (c) there is no interaction effects between both factors,

Subsequently, identify a suitable random effects model, study model residual behavior and draw relevant conclusions.

Pr. 6.4

The thermal efficiency of solar thermal collectors decreases as their average operating temperatures increase. One of the means of improving the thermal performance is to use selective surfaces for the absorber plates which have the special property that the absorption coefficient is high for the solar radiation and low for the infrared radiative heat losses. Two collectors, one without a selective surface and another with, were tested at four different operating temperatures under replication r = 4. The experimental results of thermal efficiency in % are tabulated In Table 6.28.

Table 6.28 Thermal efficiencies (%) of the two solar thermal collectors. (Problem 6.4)

• (a) Perform an analysis of variance to test for significant main and interaction effects,

• (b) Identify a suitable random effects model,

• (c) Identify a linear regression model and compare your results with those from part (b),

• (d) Study model residual behavior and draw relevant conclusions.

Pr. 6.5

The close similarity between a factorial design model and a multiple linear regression model was illustrated in Example 6.3.1. You will repeat this exercise with data from Example 6.3.2.

• (a) Identify a multiple linear regression model and verify that the parameters of all regressors are identical to the factorial design model,

• (b) Verify that model coefficients do not change when multiple linear regression is redone with the reduced model using variables coded as − 1 and + 1,

• (c) Perform a forward step-wise linear regression and verify that you get back the same reduced model with the same coefficients.

Pr. 6.6

2³ factorial analysis for strength of concrete mix

A civil construction company wishes to maximize the strength of its concrete mix with three factors or variables: A—water content, B—coarse aggregate, and C—silica. A 2³ full factorial set of experimental runs, consistent with the nomenclature of Table 6.13, was performed. These results are assembled below:

[ 58.27, 55.06, 58.73, 52.55, 54.88, 58.07, 56.60, 59.57]

• (a) You are asked to analyze this data so as to identify statistically meaningful terms,

• (b) If the minimum and maximum range of the three factors are: A(0.3576, 0.4392), B(0.4071, 0.4353) and C(0.0153, 0.0247), develop a prediction model for this problem,

• (c) Identify a multiple linear regression model and verify that the parameters of all regressors are identical to the factorial design model,

• (d) Verify that model coefficients do not change when multiple linear regression is redone with the reduced model.

Pr. 6.7

As part of the first step of a response surface (RS) approach, the following linear model was identified from preliminary experimentation using two coded variables

$$ y = 55 - 2.5{x_1} + 1.2{x_2}\quad{\rm{with}}\quad - {\rm{1}} \le {x_i} \le+ 1 $$

Determine the path of steepest ascent, and draw this path on a contour plot.

Pr. 6.8

Predictive model inferred from 2 ³ factorial design on a large laboratory chiller

Table 6.29 assembles steady state data of a 2³ factorial series of laboratory tests conducted on a 90 Ton centrifugal chiller. There are three response variables (T_cho—chilled water leaving the evaporator, T_cdi—cooling water entering the condenser, and Q_ch—chiller cooling load) with two levels each, thereby resulting in 8 data points without any replication. Note that there are small differences in the high and low levels of each of the factors because of operational control variability during testing. The chiller Coefficient of Performance (COP) is the response variable.

Table 6.29 Laboratory tests from a centrifugal chiller. (Problem 6.8)

• (a) Perform an ANOVA analysis, and check the importance of the main and interaction terms using the 8 data points indicated in the table,

• (b) Identify the parsimonious predictive model from the above ANOVA analysis,

• (c) Identify a least square regression model with coded variables and compare the model coefficients with those from the model identified in part (b),

• (d) Generate model residuals and study their behavior (influential outliers, constant variance and near-normal distribution),

• (e) Reframe both models in terms of the original variables and compare the internal prediction errors,

• (f) Using the four data sets indicated in the table as holdout points meant for cross-validation, compute the NMSE, RMSE and CV values of both models. Draw relevant conclusions.