Is Quantitative Research Ethical? Tools for Ethically Practicing, Evaluating, and Using Quantitative Research

Abstract

This editorial offers new ways to ethically practice, evaluate, and use quantitative research (QR). Our central claim is that ready-made formulas for QR, including ‘best practices’ and common notions of ‘validity’ or ‘objectivity,’ are often divorced from the ethical and practical implications of doing, evaluating, and using QR for specific purposes. To focus on these implications, we critique common theoretical foundations for QR and then recommend approaches to QR that are ‘built for purpose,’ by which we mean designed to ethically address specific problems or situations on terms that are contextually relevant. For this, we propose a new tool for evaluating the quality of QR, which we call ‘relational validity.’ Studies, including their methods and results, are relationally valid when they ethically connect researchers’ purposes with the way that QR is oriented and the ways that it is done—including the concepts and units of analysis invoked, as well as what its ‘methods’ imply more generally. This new way of doing QR can provide the liberty required to address serious worldly problems on terms that are both practical and ethically informed in relation to the problems themselves rather than the confines of existing QR logics and practices.

Appendix

Typical regression methods minimize the residual variance of outcome variables by predicting the mean (or statistical ‘expectation’) of an outcome. This can be shown by a simple regression model as follows:

$$y_{i} = a + \beta x_{i} + e_{i}$$

wherein $y_{i}$ is an outcome for some unit i, $a$ is a regression intercept, $\beta$ is a slope linking a predictor $x_{i}$ to the outcome, and $e_{i}$ is a residual. Typical regression assumptions pertain to $e$ because this is parameterized as a random variable for estimation and inference, typically with a normal distribution such that:

$$e\sim\,N\left( {0,\;\sigma^{2} } \right),$$

wherein the residual variable has zero mean and variance $\sigma^{2}$.

However, if the outcome variable y is parameterized using the regression equation, the prediction of the outcome enters as the variable’s average. Specifically:

$$y\sim\,N\left( {a + \beta x,\;\sigma^{2} } \right)$$

wherein all terms are as before, but the focus on the average of the outcome $y$ at each level of the predictor $x$ is clarified by showing how what is predicted are average levels of the outcomes $y$ at different values of the predictor $x$.

The implication is that most regression methods implicitly assume that predicting averages are what is of greatest interest to researchers. With a focus on reducing errors in inference, the best way to do this probabilistically is to predict averages, but this is only true to the extent that a single numerical prediction of an assumedly homogenous group is desired based on the group’s average standing along an outcome $y$ at a specific value of a predictor $x$. However, whether or not (and to what extent) averages may be relevant for a specific purpose and research orientation is typically left unclarified in QR, and we propose that this should be examined on a case-by-case basis with an eye to the ethics this or other QR practices.