Commit 1bc29c2
authored
feat(airt): model-extraction & membership-inference attack generators (#104)
* fix(ai-red-teaming): import get_generator in generated agentic script
Same missing-import bug as the ATLAS template (#100), in the separate
`_build_agentic_imports`: generated agentic-attack scripts call
get_generator (proxy-routing block) but never imported it, raising
NameError at runtime. Add the import and a TestAgenticGeneration
regression test (the agentic generator was previously untested).
* feat(airt): add model-extraction & membership-inference attack generators
Adds generate_extraction_attack and generate_membership_attack to attack_runner.
They build workflow scripts that construct a PredictionTargetSpec from a target's
predict endpoint (or an environment challenge_url) and run the SDK's extraction
(equation_solving/jacobian/copycat/knockoff) or membership (threshold/label_only)
attacks under an Assessment, exporting airt spans to the platform.
Verified: a generated knockoff-extraction workflow ran against a live fraud target
(fidelity 0.93, 500 queries) and completed its assessment on the local platform.
* feat(airt): expose extraction/membership attacks as agent tools
Add generate_extraction_attack and generate_membership_attack @safe_tool wrappers so
the AIRT agent (and TUI) can run the new traditional-ML privacy attacks, mirroring the
existing generate_image_attack wrapper over the attack_runner generators.
* feat(airt): add model-evasion attack generator and bump to 1.9.0
Wire the boundary (numeric L2/Linf decision search) and text (token-flip)
evasion attacks into the workflow runner and expose generate_evasion_attack
as an agent tool. Bump capability to 1.9.0 and note traditional black-box ML
coverage (extraction, membership inference, evasion) in the description.
* feat(airt): add deepwordbug evasion strategy to generator
* feat(airt): expand traditional-ML attack roster to 1.10.0
Register the new named attacks in the generator dispatch maps: evasion
(hopskipjump, simba, square, zoo, textfooler), extraction (activethief,
distillation), and membership (lira, shadow_model, entropy, loss).
* docs(airt): advertise new attack types in generator tool descriptions
Update the attack_type parameter descriptions so the agent surfaces the new
evasion (hopskipjump/simba/square/zoo/textfooler), extraction (activethief/
distillation), and membership (entropy/loss/shadow_model/lira) options.1 parent 95951f8 commit 1bc29c2
3 files changed
Lines changed: 705 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
12 | | - | |
13 | | - | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
14 | 16 | | |
15 | 17 | | |
16 | 18 | | |
| |||
0 commit comments