Membership inference (safe, educational)

Measure how overfitting reveals whether a record was in the training data.

AdvancedPrivacy Lab
العربية

The idea

A model is trained on synthetic data; a loss-threshold attack (Yeom et al.) is applied and its advantage over guessing measured.

Data description

Synthetic data generated in your browser from stated distributions; no real personal data.

Suggested next experiment

Concepts in this experiment

Membership advantage formula

0 means the attack is no better than guessing.

\mathrm{Adv}=\mathrm{TPR}_{\text{attack}}-\mathrm{FPR}_{\text{attack}}
Membership inference

Trying to tell whether a record was in a model's training data. The simplest form (Yeom et al., 2018) thresholds the loss: an overfit model has lower loss on its training examples. Risk is measured by advantage = TPR − FPR; regularisation and DP reduce it. In the lab it runs on synthetic data only.

Scientific sources for this experiment

  1. Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting (arXiv:1709.01604)S. Yeom, I. Giacomelli, M. Fredrikson, S. Jha (2018) — Paper
  2. Membership Inference Attacks against Machine Learning Models (arXiv:1610.05820)R. Shokri, M. Stronati, C. Song, V. Shmatikov (2017) — Paper

More in this module