How to Generate Unbiased Data

Abstract

One motto of our times could be "data is the new gold"—however, it will shine only if it is pure and free of dirt. Biased data can be lethally polluted and thus worthless. For example, a tax authority once asked me to help them build an algorithm to direct customs inspectors to those containers in the port that were most likely to contain contraband. The project could not go ahead because the only data they had was from a very limited number of customs inspections their officers had done in the past year. The problem: Customs inspectors had chosen which containers to check, and they had complete freedom in how to conduct the checks (e.g., they may have limited themselves to opening the first box falling into their hands and accepting the shipment as containing "Louis Vuitton Croisette handbags" because the duffel bags with a "Luis Vitton" label seemed close enough, or they may have completely emptied the container and carefully checked the L’s, O’s, and T’s of the Louis Vuitton stamp of two dozen bags, knowing that variations in these letters are among the frequent tell-tale signs of fake bags).