Assessing Phase Diagram Accuracy
 95 Downloads
Abstract
Assessing the predictive power of any computational model requires the definition of an appropriate metric or figureofmerit (e.g. mean square error, maximum error, etc). However, quantifying errors in an alloy phase diagram with a single figureofmerit is a considerably more complex problem. The “distance” between phase boundaries is not a uniquely defined concept and different phase diagrams may differ in the possible stable phases which they predict, making it unclear which “distance” to measure. Given the difficulty associated with such metrics, we instead propose to use differences in predicted phase fractions between different phase diagrams as the basis of a suitable metric. We prove that our criterion satisfies all the properties of the mathematical notion of a norm or of a metric, in addition to other properties directly relevant to phase stability problems. We illustrate the use of such criterion to the study of the convergence of assessments performed on the same alloy system by different authors over time.
Keywords
figureofmerit materials informatics metric norm1 Introduction
One of the underlying assumption of the goal of achieving “predictive science” is the availability of suitable metric to quantify the accuracy of the predictions. Although mean square errortype quantities are often wellsuited for this purpose when the predicted quantities are simple functions or vectors, quantifying errors in a more general graph, such as phase diagram, in a single figureofmerit represents a considerably more complex problem. Given this, it is perhaps not surprising that the area of research focusing on the construction of thermodynamic models [often referred to as CALculation of PHAse Diagram (CALPHAD)][5,10, 11, 12,15] is currently lacking a theoretically justified and widely adopted figureofmerit to quantify the discrepancies between two possible phase diagrams obtained via different routes. This paper intends to fill this gap by building upon earlier proposals.[17]
Given these issues, we instead propose to quantify the differences between two phase diagrams via differences in the predicted phase fractions of the corresponding phases. Phase fractions have been put forward as powerful and fundamental descriptors of phase equilibria.[16] Phase fractions are scalar, dimensionless, everywhere defined and merely take the value 0 when a phase is not stable. These desirable properties solve all of the aforementioned problems. We prove that a criterion based on phase fractions satisfies all the properties of the mathematical notion of a norm or of a metric, in addition to other properties directly relevant to phase stability problems. We illustrate the use of such a criterion to the study of the convergence of assessments performed on the same alloy system by different authors over time.
2 Definition and Motivation
Let \(f\left( c\right) =\left( f_{1}\left( c\right) ,\ldots ,f_{p}\left( c\right) \right) ^{T}\) denote a pdimensional vector of the phase fractions of all possible phases in the system under conditions \(c=\left( x,T\right) \), where T is temperature and x is a vector of overall compositions (omitting one composition, to avoid redundancy). One could also include pressure into the vector c, if desired. The knowledge of this vectorvalued function over some region R fully defines the phase diagram over that region.
Since this definition is mathematically equivalent to a socalled weighted \( L_{1}\) norm[18] defined on a vectorvalued field, it automatically inherits all the natural properties of norm: It is zero if only if the two phase diagrams \(f^{1}\) and \(f^{2}\) agree,^{1} it is symmetric (\( \left\ f^{1}f^{2}\right\ _{R}=\left\ f^{2}f^{1}\right\ _{R} \)) and it obeys the triangular inequality \(\left\ f^{1}f^{2}\right\ _{R}\le \left\ f^{1}f^{3}\right\ _{R}+\left\ f^{3}f^{2}\right\ _{R}\), for 3 phase diagrams \(f^{1},f^{2},f^{3}\). (A norm also satisfies \(\left\ af\right\ =\left a\right \left\ f\right\ \) but this property is not useful in this context, since the phase fractions must sum up to one.) Since a norm is a special case of a metric, our proposal also defines a proper metric.
Definition (1) provides a dimensionless quantity, which facilitates its interpretation. Another desirable property is that it naturally handles the case when one phase is simply missing in one phase diagram. This possibility is not uncommon when comparing experimental and ab initio phase diagrams. Also, in novel systems that are not yet well characterized, there may not be perfect knowledge of which phases are stable or metastable and it is useful to be able to quantify this type of discrepancy. It is not clear how missing phases could be handled with a figureofmerit based on distances between phase boundaries. This definition also applies a less severe penalty in situations in which only one phase is in disagreement while the others agree. This makes sense, since this situation typically arises when two phases have very similar free energies and can easily be mispredicted without affecting the reliability of the phase diagram elsewhere.
Our approach also makes it simple to account for the fact that Gibbs triangles (or, more generally, Gibbs simplexes) are the most natural way to represent multiple composition axes. The fact that the axes are not orthogonal can be ignored in definition (1) because the same Jacobian terms appear in both the numerator and denominator. One can thus simply integrate over all but one composition using orthogonal axes, whether these axes are truly orthogonal or not in the phase diagram’s representation.
The definition does exhibit some limitations. Most importantly, it is dependent on the choice of the region of interest R. This could be mitigated by agreeing on standardized regions. For instance, one can use a region including the entire composition range and temperatures from absolute zero (or room temperature) to the highest phase transformation temperature of all systems considered.
The presence of a miscibility gap leads to a subtle complication in Definition (1) [or (4)]. In this case, multiple phases exhibiting the same crystal structure but at different compositions could be in a multiphase equilibrium. We handle this by considering each phase (even with the same crystal structure) as distinct but when comparing the resulting phase fractions across two phase diagrams, we always reorder the phase fractions (among phases sharing the same crystal structure) so as to minimize (1). If the number of phases with the same crystal structure is different in the two phase diagrams, we then add the appropriate number of phases with a zero phase fraction. In the limit where the differences between the two phases diagrams are small, this simple rule yields differences in phase fraction between corresponding phases.

phasenorm tdb1= tdbfile1 tdb2= tdbfile2 e= element1,element2,... n= nb of samples T0= min temperature T1= max temperature [01]

tdbfile1 and tdbfile2 are the two thermodynamic database files (in the TDB format[1]) of the assessments to be compared;

element1,element2,... is a commaseparated list of the elements involved in the phase diagram of interest (which allows the user to extract a subsystem from the TDB files);

nb of samples specifies the number of Monte Carlo sampling steps performed;

min temperature and max temperature define the temperature range of the region R of interest (the full composition range is assumed);

the optional 01 switch instructs the use of Eq 4 instead of (1).
When using this tool, it is important to ensure that the two thermodynamic database files use the same naming conventions for the phases.
3 Application Example
The figureofmerit proposed here enables instructive quantitative studies of the accuracy of phase diagrams. One natural question, for instance, is whether the assessments of an alloy system are actually converging, that is, becoming more accurate over time as more data because available and more researchers study the same system.
In Fig. 2, it can be seen that the distances between assessments do clearly decrease sharply over time, indicating that a consensus regarding the FeTi phase diagram is steadily emerging. In contrast, in the AlCu system (shown in Fig. 3), it appears that disagreements have persisted for many years, although the two latest assessments reported in 2016[20,22] do seem to show good mutual agreement. This analysis implicitly assumes that even if a recent study reuses older assessments, its authors consider it as the current stateoftheart, so that this data set inherits the “time stamp” of its most recent (re)use.
We can also use our metric to identify clusters of work that report mutually consistent results. Figure 4 and 5 report all pairwise distances between the assessments. For a given similarity threshold (here 1 or 2%), one can find groups of assessments that lie close to each other, within that threshold. Encouragingly, these clusters seem to primarily consist of recent publications thus again suggesting an emerging consensus.
4 Conclusion
We have described a formal methodology to quantify, in a single figureofmerit, the level of agreement between two phase diagrams. Our proposal not only satisfies the mathematical requirements of a norm or a metric, but also has a sound physical basis, is invariant to scaling of the graph axes, and is easy to compute via Monte Carlo sampling, with or without access to the thermodynamic model underlying each phase diagram. We illustrate its usefulness in a metaanalysis of a set of thermodynamic assessments in popular alloy systems, in an effort to determine whether the most current assessments have reached a consensus. Our metric may find applications in other areas as well, for instance, to report how well phase diagrams generated purely via ab initio methods agree with the corresponding experimentsbased thermodynamic assessements.
Footnotes
 1.
Except perhaps on an irrelevant set of zero measure.
Notes
Acknowledgments
This research was supported by ONR under Grant N000141712202, and by Brown University through the use of the facilities at its Center for Computation and Visualization. This work uses the Extreme Science and Engineering Discovery Environment (XSEDE) resource Stampede 2 at the Texas Advanced Computing Center through allocation TGDMR050013N, which is supported by National Science Foundation Grant Number ACI1548562. The author would like to thank Kenny Lipkowitz for suggesting this line of research.
References
 1.J.O. Andersson, T. Helander, L. Höglund, P.F. Shi, B. Sundman, ThermoCalc & DICTRA, Computational Tools for Materials Science. Calphad 26, 273 (2002)CrossRefGoogle Scholar
 2.M. Barnsley, Fractals Everywhere (Academic Press Professional Inc., San Diego, 1988)zbMATHGoogle Scholar
 3.H. Bo, L. Duarte, W. Zhu, L. Liu, H. Liu, Z. Jin, C. Leinenbach, Experimental Study and Thermodynamic Assessment of the CuFeTi System. Calphad 40, 24–33 (2013)CrossRefGoogle Scholar
 4.H. Bo, J. Wang, L. Duarte, C. Leinebach, L. bin Liu, H. shan Liu, Z. peng Jin, Thermodynamic Reassessment of FeTi Binary System. Trans. Nonferrous Met. Soc. China 22(9), 2204–2211 (2012)CrossRefGoogle Scholar
 5.C. Campbell, U. Kattner, Z.K. Liu, File and Data Repositories for Next Generation CALPHAD. Scr. Mater. 70, 7 (2014). https://doi.org/10.1016/j.scriptamat.2013.06.013 CrossRefGoogle Scholar
 6.C. Dai, H. Xu, S. Wang, M. Yin, S. Cui, L. Zhang, Y. Du, F. Zheng, Interdiffusivities and Atomic Mobilities in fcc CuAlFe Alloys. Calphad 35(4), 556–561 (2011)CrossRefGoogle Scholar
 7.L.F.S. Dumitrescu, M. Hillert, N. Sounders, Comparison of FeTi Assessments. J. Phase Equilibria 19(5), 441–448 (1998)CrossRefGoogle Scholar
 8.E. Galvan, R.J. Malak, S. Gibbons, R. Arroyave, A Constraint Satisfaction Algorithm for the Generalized Inverse Phase Stability Problem. J. Mech. Des. 139, 011401 (2017)CrossRefGoogle Scholar
 9.C. Guo, C. Li, X. Zheng, Z. Du, Thermodynamic Modeling of the FeTiV System. Calphad 38, 155–160 (2012)CrossRefGoogle Scholar
 10.T. Hickel, U.R. Kattner, S.G. Fries, Computational Thermodynamics: Recent Developments and Future Potential and Prospects. Phys. Status Solidi B 251, 9 (2014)ADSCrossRefGoogle Scholar
 11.U.R. Kattner, Thermodynamic Modeling of Multicomponent Phase Equilibria. JOM J. Min. Met. Mat. S. 49, 14 (1997)CrossRefGoogle Scholar
 12.L. Kaufman, Computational Thermodynamics and Materials Design. Calphad 25, 141 (2001)CrossRefGoogle Scholar
 13.K. Kumar, P. Wollaiits, L. Delaey, Thermodynamic Reassessment and Calculation of FeTi Phase Diagram. Calphad 18(2), 223–234 (1994)CrossRefGoogle Scholar
 14.S.M. Liang, R. SchmidFetzer, Thermodynamic Assessment of the AlCuZn System, Part II: AlCu Binary System. Calphad 51, 252–260 (2015)CrossRefGoogle Scholar
 15.Z.K. Liu, FirstPrinciples Calculations and CALPHAD Modeling of Thermodynamics. J. Phase Equilib. Differ. 30, 517 (2009)CrossRefGoogle Scholar
 16.J. Morral, TwoDimensional Phase Fraction Charts. Scr. Metall. 18, 407 (1984)CrossRefGoogle Scholar
 17.J. Morral, H. Gupta, A Figure of Merit for Predicted Phase Diagrams. J. Phase Equilib. 13, 373 (1992)CrossRefGoogle Scholar
 18.A.W. Naylor, G.R. Sell, Linear Operator Theory in Engineering and Science (Springer, Berlin, 1982)CrossRefzbMATHGoogle Scholar
 19.R. Otis, M. Emelianenko, Z.K. Liu, An Improved Sampling Strategy for Global Energy Minimization of MultiComponent Systems. Comput. Mater. Sci. 130, 282 (2017)CrossRefGoogle Scholar
 20.Y. Sun, H. Liu, Z. Xie, Z. Jin, Prediction of Interfacial Reaction Products Between Metals with Same Lattice Structure Through Thermodynamic Modeling. Calphad 52, 180–185 (2016)CrossRefGoogle Scholar
 21.B. Sundman, U.R. Kattner, M. Palumbo, S.G. Fries, OpenCalphad—A Free Thermodynamic Software. Integr. Mater. Manuf. Innov. 4, 1 (2015)CrossRefGoogle Scholar
 22.B. Sundman, U.R. Kattner, C. Sigli, M. Stratmann, R.L. Tellier, M. Palumbo, S.G. Fries, The OpenCalphad Thermodynamic Software Interface. Comput. Mater. Sci. 125, 188 (2016)CrossRefGoogle Scholar
 23.A. van de Walle, Multicomponent Multisublattice Alloys, Nonconfigurational Entropy and Other Additions to the Alloy Theoretic Automated Toolkit. Calphad 33, 266–278 (2009). https://doi.org/10.1016/j.calphad.2008.12.005 CrossRefGoogle Scholar
 24.A. van de Walle, Methods for FirstPrinciples Alloy Thermodynamics. JOM J. Min. Met. Mat. S. 65, 1523–1532 (2013). https://doi.org/10.1007/s1183701307643 CrossRefGoogle Scholar
 25.A. van de Walle, M.D. Asta, G. Ceder, The Alloy Theoretic Automated Toolkit: A user guide. Calphad 26, 539–553 (2002). https://doi.org/10.1016/S03645916(02)800062 CrossRefGoogle Scholar
 26.A. van de Walle, C. Nataraj, Z.K. Liu, The Thermodynamic Database. Calphad 61, 173 (2018). https://doi.org/10.1016/j.calphad.2018.04.003 CrossRefGoogle Scholar
 27.S. Wang, K. Wang, G. Chen, Z. Li, Z. Qin, X. Lu, C. Li, Thermodynamic Modeling of TiFeCr Ternary System. Calphad 56, 160–168 (2017)CrossRefGoogle Scholar
 28.V. Witusiewicz, U. Hecht, S. Fries, S. Rex, The AgAlCu System Part I: Reassessment of the Constituent Binaries on the Basis of New Experimental Data. J. Alloys Compd. 385(1–2), 133–143 (2004)Google Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.