Causality, Simpson’s Paradox, and Context-Specific Independence

Abstract

Cognitive psychologist Patricia Cheng suggests that erroneous causal inference is perhaps too often incorrectly attributed to problems with the process of inference rather than the data on which the inference is carried out. In this paper, we discuss the role of incomplete data in making faulty inferences and where those problems arise. We focus on one of two potential problems in the data we call ‘unmeasured-in’ and ‘unmeasured-out’ and address a generalization of the causal knowledge in the hope of detecting independencies hidden inside variables, causing the system to behave less than adequately.

The interpretation of the data can be more representative of the problem domain by examining subsets of values for variables in the data. We show how to do this with a generalized form of statistical independence that can resolve relevance problems in the causal model. The most interesting finding is how the examination of contexts can formalize the paradoxical statements in Simpson’s paradox and how a simple detection method can eliminate the problem.