A multiobjective Gaussian process approach for optimization and prediction of carbonization process in carbon fiber production under uncertainty
 17 Downloads
Abstract
During composite fiber production, carbon fibers are normally derived from polyacrylonitrile precursor. Carbonization, as a key step of this process, is significantly energyconsuming and costly, owing to its high temperature requirement. A costeffective approach to optimize energy consumption during the carbonization is implementing predictive modeling techniques. In this article, a Gaussian process approach has been developed to predict the mechanical properties of carbon fibers in the presence of manufacturing uncertainties. The model is also utilized to optimize the fiber mechanical properties under a minimum energy consumption criterion and a range of process constraints. Finally, as the Young’s modulus and ultimate tensile strength of the fibers did not show an evident correlation, a multiobjective optimization approach was introduced to acquire the overall optimum condition of the process parameters. To estimate the tradeoff between these material properties, the standard as well as an adaptive weighted sum method were applied. Results were summarized as design chart for potential applications by manufacturing process designers.
Keywords
Carbon fiber Gaussian process Carbonization Optimization Adaptive weighted sum method Machine learning1 Background and introduction
Carbon fibers, a lightweight, highstiffness, and highstrength material type, have found wide interest and applications in leading industries such as aerospace, automotive, and energy [1]. Over the past few decades, the quality of carbon fibers has been continually enhanced using newly developed precursors and processes. In particular, much research and effort have been put into improving the tensile properties of fibers, and simultaneously optimizing the costs and energy consumption of carbon fiber production processes [2]. However, the existence of uncertainties in carbon fiber manufacturing can still negatively affect the performance and robustness of the final product and, accordingly, the analysis and decisionmaking steps for designers. The random behavior of hundreds of thousands of filaments in the production line is one of the primary causes of uncertainty [3, 4], which if not accounted for, it can make the design analysis of carbon fiber production complex, timeconsuming, and costly. Yet, the implementation of a powerful and flexible stochastic process model can facilitate the tasks of prediction and optimization of carbon fiber production for designers [5].
Gaussian process (GP) is a useful probabilistic inference method, which effectively deals with complex data structures in the presence of uncertainty [6]. GP provides a robust prediction model and avoids overfitting [7, 8]. Moreover, GP has been successfully implemented in numerous engineering case studies and has proved to provide more reliable and robust predictions compared with classic regression methods [9, 10, 11]. Consequently, GP may be deemed as an appropriate statistical framework for the prediction and optimization of carbon fiber production processes under uncertain data structures.
1.1 Carbonization process
In the first stage of the carbonization process, the lowtemperature (LT) furnace heats the stabilized PAN fibers up to 950 °C [14]. The LT furnaces usually consume electricity for generating the required heat. The temperature increases gradually by a rate of 5 °C/min in order to lessen the mass transfer, which is primarily due to lowfiber formation capacity. One primary purpose of this stage is to remove tars from the fibers. Temperatures higher than 1000 °C can result in the decomposition of tars, which cause inadequate modifications in the fibers’ properties [2].
The temperature in the carbonization process plays a crucial role in the characterization of the carbon fibers’ mechanical properties, and it must be selected according to the resulting fibers’ application [2]. In other words, the tensile properties of the carbon fibers vastly depend on the production temperature. By increasing the process temperature, a steady growth in the modulus value can be observed [16]. A temperature of 1000 °C produces a low modulus outcome, whereas 1500 °C can result in intermediate modulus carbon fibers (type II carbon fiber) [16, 17]. Furthermore, in order for carbon fibers to reach a higher modulus, higher temperatures, 1600–1800 °C and up to 3000 °C, are necessary [17, 18]. The latter, in which the fibers are heated up to 3000 °C, is called graphitization. Trinquecoste et al. [19] also suggested that, although high tensile strength can be attained by a temperature around 1500 °C, more heating treatment is required to achieve a higher modulus.
1.2 Stateoftheart on modeling of carbon fiber production process
Since the carbon fiber production process is extremely expensive and timeconsuming, the preference of the industrial sectors is to minimize the number of the process runs during data collection. The limited number of experimental data and severe uncertainty makes the use of mathematical modeling in the carbon fiber production process highly essential for both process design and optimization purposes. Very recently, different mathematical methods, namely, nonlinear multivariable regression, LevenbergMarquardt neural network algorithm, thin plate splines, and convex hull technique, were implemented to optimize the mean value of the resulting carbon fiber modulus [20]. However, the variability of the carbon fiber mechanical properties was not taken in to account. In this paper, a GP modeling approach for the prediction and, consequently, optimization of the carbonization process is implemented. The existence of excessive amount of noise and potential autocorrelation between data points, which can be emanated due to the continuous or multistage timedependent nature of the processes, are the primary reasons for utilizing GP.
2 Gaussian process
Machine learning is a category of artificial intelligence that is capable of developing statistical models to interpolate, extrapolate, and learn a pattern without the interaction of humans. GP is an effective tool in machine learning; in fact, many models that are used in the field of machine learning are specific types of GPs [7]. GP, a powerful probabilistic method, can cope with the uncertainty. This reliable method provides robust and accurate stochastic prediction models in the presence of noise [21] based on Baye’s rule. One significant benefit of GPs is that, unlike regression methods, it is not necessary to provide a predefined explicit relation between inputs and outputs. GPs establish the required relations in an unsupervised framework based on the properties of the prior functions and the given experimental evidence [7]. Moreover, GPs favorably predict a fit error at each prediction point separately, as opposed to the standard regression methods which output a single overall fit error.
2.1 Gaussian process basics
2.2 Covariance functions
From an unsupervised learning point of view, the similarity is a fundamental concept underlying the prediction of test points [7]. It implies that points with close xs tend to have similar target values, ys. In GP, the covariance function is responsible for specifying this similarity and closeness of the data points. In fact, the covariance functions govern the properties of the GP prediction model.
3 Objective: model development
The focus of this research is to incorporate the data collected from carbonization processes in [15] (see also Appendix for raw experimental data) into the training of machine learning algorithms under uncertanity and then use the predicted values to address the process optimization problem. In other words, the objective here is to determine how a robust GP framework can predict and optimize the mechanical properties of carbon fibers as well as the process’s energy consumption under different processing parameters and bounds.
3.1 Carbonization process furnace
The data that is being investigated in this paper was collected from the carbonization processes performed as a part of the carbon fiber production process at the Carbon Nexus Institute at Deakin University in Melbourne, Australia [15]. The multizoned carbonization furnace at Carbon Nexus Institute provides a low temperature between 1100 and 1175 °C, and a high temperature between 1400 and 1475 °C. The schematic figure of the furnace is provided in Fig. 2. These temperatures are modified by altering the manageable furnaces’ heat elements. The pure nitrogen gas, which exists in the inert atmosphere of the furnaces, is responsible for removing the gasses evolved in the furnace during the carbonization process. The Process N_{2} valve (PN) provides enough nitrogen gas to be employed in the furnace. The ratio of nitrogen supplement can vary between 13 and 24 l per min \( \left(\frac{l}{\min}\right) \) [15]. The adjustable speed of the drivers (DR4 and DR5) allows for the application of the required tension to the fibers. The drivers’ speed varies from 25.4 to 28.9 m h \( \left(\frac{m}{h}\right) \). During the carbonization stage, the precarbonized fibers pass through the LT and HT furnaces during a specific amount of time and under a predetermined tension. These process conditions are selected based on the desired final mechanical properties, e.g., as defined by industrial application designers.
3.2 A black box problem
Due to the large amount of uncertainty that exists in carbon fiber production, it is almost impossible to explicitly determine the relation between inputs and outputs of the carbonization process. Therefore, any prediction or optimization investigation would primarily fall into the black box problem category. A black box is a system in which no information and explicit mathematical understanding of the inner function of the system is derivable [25]. The only attainable knowledge is the characteristics of the system evidenced through input and output data. One of the primary benefits of GP is that it can appropriately deal with black box problem scenarios as it does not require supervised modeling of inputoutput relations; it automatically establishes the relations based on the statistical nature of collected data (i.e., their distances and autocorrelations) in the presence of noise.
3.3 Gaussian process model development
The initial step in the proposed optimization of the carbonization process is to develop a GP prediction model for each output (Young’s modulus and tensile strength). The measured tensile properties of fibers in the carbonization process include both the mean and standard deviation (STD). The STDs under each process condition are inconsistent and have been clearly influenced by different heat treatment duration and furnace temperatures. Statistically, a state like this, in which the variability of random variables are not equal, is called heteroscedasticity [26, 27]. However, in the standard GP, the model assumes that standard deviations of actual data are alike (homoscedasticity). Hence, for heteroscedastic data, two separate GP models should be developed. The first model tackles the mean prediction. In the second GP model, the inputs are identical with the first GP model; however, the output response is now the standard deviation. Therefore, five GP models are defined in order to predict the tensile properties of resulting carbon fibers, their variabilities, and the process energy consumption.
Statistical measures of the carbonization process’s Gaussian process (GP) prediction models
GP prediction model  Covariance function  Mean function  RSS (GPa)  Rsquared 

Mean modulus  Squared exponential  Linear  0.161  0.99 
Modulus STD  Squared exponential  Linear  0.258  0.99 
Mean tensile strength  Squared exponential  Linear  0.00005  1.00 
Tensile strength STD  Squared exponential  Linear  0.0000038  1.00 
Energy consumption  Squared exponential  Constant  0.028  0.98 
4 Multiobjective optimization of carbonization process
Every process optimization problem falls into one of the two general categories, single or multiobjective. In a singleobjective optimization, the goal is to find one solution that usually minimizes or maximizes the objective in the presence of some constraints or process bounds. In a multiobjective optimization, however, there are more than one objective to be satisfied and they typically conflict one another, which results in obtaining more than one optimal (Pareto) solution. Each solution would favor one objective more or less than the other(s), while an overall (compromised) optimal solution would be desired to fit the target application. A common practical strategy in solving multiobjective problems is to create different Pareto solutions (tradeoff curve) using multicriteria decisionmaking techniques [31], and, accordingly, the decisionmaker can select the final optimal solution based on their overall preference, additional practical constraints, and bounds, etc.
4.1 Constraints and bounds

Zone 1’s temperature interval is from 1100 to 1175 °C: 1100 < T_{1}(°C) < 1175

Zone 2’s temperature interval is from 1400 to 1475 °C: 1400 < T_{2}(°C) < 1475

The amount of time that the fibers spend in the furnaces is between 151 and 167 s: 151 < t(s) < 167.
More discussions and example on the “applicationoriented” constraints will be given in the next section.
4.2 A multiobjective optimization scenario
High modulus and high tensile strength are the primary mechanical properties of interest for carbon fiber applications. However, several timeconsuming and expensive carbon fiber manufacturing tests must be performed to find such optimal properties. In addition, as discussed in Section 1, the experiments are highly influenced by uncertainty factors such as vibrations within thousands of filaments in each tow, as operator skills, etc. An alternative is the use of black box models to predict the optimum manufacturing conditions for producing carbon fibers with high modulus and tensile strength. Herein, the weighted sum method is implemented to find the overall optimal mean values of both properties. In this method, the weights (w_{j}) of normalized decision variables (mean elastic modulus and mean ultimate tensile strength/UTS) are chosen based on the importance of each objective per preference of the decisionmaker (e.g., industry designer) [34]. The normalized values is found by dividing the modulus and tensile strength values by their respective maxima. The STD of the mechanical properties and the value of energy consumption are deemed as secondary decision variables in this scenario and were converted to constraints as follows:
Objective function: maximize w_{1}Modulus_{norm}+ w_{2}UTS_{norm},

The standard deviation of the modulus to be less than 10 GPa: STD_{modulus} < 10

The standard deviation of the tensile strength to be less than 0.9 GPa: STD_{UTS} < 0.9

The consumed energy to be less than 12.45 kWh: E (kWh) < 12.45
Relative weights (preference values) w_{1} and w_{2} are specific to the designer’s needs.
Pareto optimal solutions using different weights
Modulus weight  Tensile strength weight  Temperature of zone 1 (°C)  Temperature of zone 2 (°C)  Time (s)  Modulus (GPa)  Tensile strength (GPa)  Modulus STD (GPa)  Tensile strength STD (GPa)  Energy consumption (kWh) 

0.1  0.9  1124  1453  156  216.80  3.082  9.26  0.99  12.43 
0.2  0.8  1126  1455  157  217.78  3.081  9.99  0.96  12.45 
0.3  0.7  1124  1454  157  218.15  3.079  9.96  0.90  12.44 
0.4  0.6  1120  1457  158  221.42  3.056  9.97  0.39  12.44 
0.5  0.5  1118  1457  158  222.09  3.049  9.79  0.25  12.43 
0.6  0.4  1114  1454  158  222.54  3.041  9.89  0.20  12.41 
0.7  0.3  1114  1454  158  222.54  3.041  9.89  0.20  12.41 
0.8  0.2  1113  1453  158  222.58  3.04  9.95  0.21  12.41 
0.9  0.1  1113  1453  158  222.58  3.04  9.95  0.21  12.41 
4.3 Adaptive weighted sum method
5 Discussions
5.1 The effect of stabilization process temperature on the carbon fiber mechanical properties
It was discussed that the furnace temperature during the carbonization process can closely affect the mechanical properties of the resulting carbon fibers. Early research on the carbon fibers has revealed that the fibers’ modulus has a strong correlation with the process temperature [2]. The experimental data of the current carbonization process, however, states an opposite behavior, such that the carbon fibers’ mean modulus has decreased as the furnace temperatures increase. In other words, the highest mean modulus occurs when the zone1 temperature is 1100 °C (lowest value). This contrary behavior is believed to have originated from the gap between the furnace temperatures of the stabilization and carbonization process. The process of heating the PAN precursor is highly exothermic. Unless the increase in temperature of fibers is slow and gradual, the thermal runaway reaction will take place and heating process will become uncontrollable, resulting in fibers getting burnt and ending up with poor mechanical properties [2].
5.2 The benefit of using a twozone furnace in the carbonization process
5.3 Recommended design chart for multiobjective optimization of carbonization process
6 Conclusion
This paper presented the application of GP in the prediction and optimization of carbon fiber production process. It was demonstrated that GP is an effective and robust tool for capturing the behavior of complex manufacturing processes under noisy responses.
The optimum covariance functions and hyperparameters of the GP model have been specified using NML and PRESS. The overall performance of the selected models was evaluated using statistical techniques such as RSS and Rsquared, to measure the closeness of the predicted values to the experimental data. The assessment results of the models clearly confirm that GPs are able to successfully capture the carbonization process response variables with goodness of fit values over 99% under highly fluctuating data, such as modulus and tensile strength STDs.
In the optimization stage of the research, the weighted sum method was adapted and applied under a process design scenario based on different assumed needs of carbon fiber customers. The standardweighted sum method was employed for the Pareto optimization of the carbon fibers’ modulus and the tensile strength. However, it was concluded that this method does not capture all of the Pareto optimal solutions, regardless of how the weights are assigned. Hence, a new approach, called the AWS method, was adapted; the results of which indicated that the model is capable of capturing all of the hidden Pareto optimal solutions in the decision space of carbonization process.
Notes
Acknowledgments
The support of colleagues and the stimulating discussions at the Composites Research Network and Carbon Nexus Institute are greatly valued.
Funding information
This study was financially supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
References
 1.Chand S (2000) Review carbon fibers for composites. J Mater Sci 35(6):1303–1313CrossRefGoogle Scholar
 2.Morgan P (2005) Carbon fibers and their composites. CRC press, Boca RatonGoogle Scholar
 3.Park SJ (2015) Carbon Fibers. Springer, DordrechtGoogle Scholar
 4.Manocha LM, Bahl OP, Jain GC (1978) Length changes in PAN fibres during their pyrolysis to carbon fibres. Die Angewandte Makromolekulare Chemie: Applied Macromolecular Chemistry and Physics 67(1):11–29CrossRefGoogle Scholar
 5.Khayyam H, Naebe M, Zabihi O, Zamani R, Atkiss S, Fox B (2015) Dynamic prediction models and optimization of polyacrylonitrile (PAN) stabilization processes for production of carbon fiber. IEEE Transactions on Industrial Informatics 11(4):887–896CrossRefGoogle Scholar
 6.Titsias M (2009) Variational learning of inducing variables in sparse Gaussian processes. Artificial Intelligence and Statistics 12:567–574Google Scholar
 7.Rasmussen CE (2004) Gaussian Processes in Machine Learning. In: Bousquet O, von Luxburg U, Rätsch G (eds) Advanced Lectures on Machine Learning. ML 2003. Lecture Notes in Computer Science, vol 3176. Springer, Berlin, Heidelberg, pp 63–71Google Scholar
 8.Hensman J, Fusi N, Lawrence ND (2013) Gaussian processes for big data, arXiv preprint arXiv:1309.6835Google Scholar
 9.Petelin D, Grancharova A, Kocijan J (2013) Evolving Gaussian process models for prediction of ozone concentration in the air. Simul Model Pract Theory 33:68–80CrossRefGoogle Scholar
 10.Gramacy RB, Haaland B (2016) Speeding up neighborhood search in local Gaussian process prediction. Technometrics 58(3):294–303CrossRefGoogle Scholar
 11.Corrado N, Gherlone M, Surace C, Hensman J, Durrande N (2015) Damage localisation in delaminated composite plates using a Gaussian process approach. Meccanica 50(10):2537–2546CrossRefGoogle Scholar
 12.Park SJ, Heo GY (2015) Precursors and manufacturing of carbon fibers. In: Carbon Fibers. Springer Series in Materials Science, vol 210. Springer, Dordrecht, pp 31–66Google Scholar
 13.Peebles LH (1994) Carbon fibres: structure and mechanical properties. Int Mater Rev 39(2):75–92CrossRefGoogle Scholar
 14.Zhu D, Xu C, Nakura N, Matsuo M (2002) Study of carbon films from PAN/VGCF composites by gelation/crystallization from solution. Carbon 40(3):363–373CrossRefGoogle Scholar
 15.Khayyam H, Naebe M, BabHadiashar A, Jamshidi F, Li Q, Atkiss S, Buckmaster D, Fox B (2015) Stochastic optimization models for energy management in carbonization process of carbon fiber production. Appl Energy 158:643–655CrossRefGoogle Scholar
 16.Ko TH (1991) The influence of pyrolysis on physical properties and microstructure of modified PAN fibers during carbonization. J Appl Polym Sci 43(3):589–600CrossRefGoogle Scholar
 17.Mittal J, Mathur RB, Bahl OP (1997) Post spinning modification of PAN fibres—a review. Carbon 35(12):1713–1721CrossRefGoogle Scholar
 18.Yusof N, Ismail AF (2012) Post spinning and pyrolysis processes of polyacrylonitrile (PAN)based carbon fiber and activated carbon fiber: a review. J Anal Appl Pyrolysis 93:1–13CrossRefGoogle Scholar
 19.Trinquecoste M, Carlier JL, Derré A, Delhaes P, Chadeyron P (1996) High temperature thermal and mechanical properties of high tensile carbon single filaments. Carbon 34(7):923–929CrossRefGoogle Scholar
 20.Khayyam H, Fakhrhoseini SM, Church JS, Milani AS, BabHadiashar A, Jazar RN, Naebe M (2017) Predictive modelling and optimization of carbon fiber mechanical properties through high temperature furnace. Appl Therm Eng 125:1539–1554CrossRefGoogle Scholar
 21.Yuan J, Wang K, Yu T, Fang M (2008) Reliable multiobjective optimization of highspeed WEDM process based on Gaussian process regression. Int J Mach Tools Manuf 48(1):47–60CrossRefGoogle Scholar
 22.QuiñoneroCandela J, Rasmussen CE (2005) A unifying view of sparse approximate Gaussian process regression. J Mach Learn Res 6(Dec):1939–1959Google Scholar
 23.Barber D (2012) Bayesian reasoning and machine learning. Cambridge University Press, CambridgeGoogle Scholar
 24.Wilson A, Adams R (2013) Gaussian process kernels for pattern discovery and extrapolation. In: International Conference on Machine Learning (ICML), JMLR W&CP 28(3):1067–1075Google Scholar
 25.Droste S, Jansen T, Wegener I (2002) A new framework for the valuation of algorithms for blackboxoptimization. FOGA 3:253–270Google Scholar
 26.Breusch TS, Pagan AR (1979) A simple test for heteroscedasticity and random coefficient variation. Econometrica: Journal of the Econometric Society 47:1287–1294Google Scholar
 27.Engle RF (1982) Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econometrica: Journal of the Econometric Society 50:987–1007Google Scholar
 28.Tarpey T (2000) A note on the prediction sum of squares statistic for restricted least squares. Am Stat 54(2):116–118Google Scholar
 29.Morgan JA, Tatar JF (1972) Calculation of the residual sum of squares for all possible regressions. Technometrics 14(2):317–325CrossRefGoogle Scholar
 30.Cameron AC, Windmeijer FA (1997) An Rsquared measure of goodness of fit for some common nonlinear regression models. J Econ 77(2):329–342CrossRefGoogle Scholar
 31.Rardin RL, Rardin RL (1998) Optimization in operations research. Prentice Hall Upper Saddle River, NJGoogle Scholar
 32.Zadeh L (1963) Optimality and nonscalarvalued performance criteria. IEEE Trans Autom Control 8(1):59–60CrossRefGoogle Scholar
 33.Gass S, Saaty T (1955) The computational algorithm for the parametric objective function. Naval research logistics quarterly 2(1–2):39–45CrossRefGoogle Scholar
 34.Censor Y (1977) Pareto optimality in multiobjective problems. Appl Math Optim 4(1):41–59CrossRefGoogle Scholar
 35.Kim IY, de Weck OL (2005) Adaptive weightedsum method for biobjective optimization: Pareto front generation. Struct Multidiscip Optim 29(2):149–158CrossRefGoogle Scholar
 36.Das I, Dennis JE (1998) Normalboundary intersection: a new method for generating the Pareto surface in nonlinear multicriteria optimization problems. SIAM J Optim 8(3):631–657CrossRefGoogle Scholar
 37.Kim IY, De Weck OL (2006) Adaptive weighted sum method for multiobjective optimization: a new method for Pareto front generation. Struct Multidiscip Optim 31(2):105–116CrossRefGoogle Scholar