Data-dependent analysis

Data-dependent analysis, or adaptive data analysis, is the problem of how to perform valid statistical inference when the analysis itself depends on the data. The problem has often been motivated by the replication crisis in science.

General approaches include:

• Worst-case simultaneous inference (e.g., PoSI)
• Selective inference, as in Jonathan Taylor’s group at Stanford
• Differential privacy