Member-only story
A Placebo Can Pass and Still Be as Large as the Estimate
I ended an article on matching with two conjectures about propensity analyses in companies. Public retail data made it possible to test both. Neither survived as written.
Diogo Ribeiro is a mathematician, researcher and lecturer at the Faculdade de Media Artes e Design, Universidade Técnica do Porto. ORCID 0009–0001–2022–7072. GitHub diogoribeiro7. Gitlab diogoribeiro7.
In “Matching Does Not Create Ignorability” I closed with two conjectures and labelled them as such because I had not tested them. The first was that in commercial data the strongest predictors of treatment are disproportionately instrument-like. My reasoning was that who receives an offer is often decided by an organisational process, and that the variables recording that process would predict treatment far better than they predict the outcome. If so, a workflow that chooses propensity features for their predictive power would favour the variables that ought to be left out. The second was that a placebo on the pre-period outcome, meaning the whole analysis rerun on an outcome the treatment cannot have affected, would fail in a large share of the propensity analyses run inside companies.
Claims about what happens inside companies seem to require a company’s data. A usable substitute is public. dunnhumby’s “The Complete Journey” follows some 2,500 households of a grocery retailer and…
