run_membership_inference
Evaluates a model's privacy risk by running a membership inference attack, returning accuracy and count of identifiable training members.
Instructions
Run a shadow-model membership-inference attack against a registered model.
`member_pool` must be rows known to have been in the model's
training set -- it trains the shadow models and supplies the
known-member evaluation sample. `nonmember_pool` must be rows
known NOT to have been in training (e.g. a held-out test split) --
it supplies the known-non-member evaluation sample only. Each pool
is capped at 5,000 samples; shadow-model training does not scale
past this in the validated protocol.
Returns the attack's accuracy/AUROC at distinguishing members from
non-members, a privacy-risk tier, and a direct count of how many
of the evaluated members would be identifiable -- not a
population-scale extrapolation.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| seed | No | ||
| n_eval | No | ||
| member_pool | Yes | ||
| model_handle | Yes | ||
| nonmember_pool | Yes | ||
| n_shadow_models | No | ||
| shadow_model_size | No |