PART 6 · MODULE 10Copying AI models
Model Stealing & Extraction
Extracting model weights and behavior
Loading module...
Extracting model weights and behavior
Model stealing (or model extraction) is the process of recreating a proprietary model by repeatedly querying it. The attacker sends carefully chosen inputs, records the outputs, and trains a copy that behaves almost identically.
Think of it as reverse-engineering a secret recipe by tasting the dish many times and adjusting your own recipe until it matches.
Model Extraction Attack Pipeline
The attacker sends thousands of inputs designed to explore the model's decision boundaries. These are not random — they systematically probe different regions of the input space.
For each input, the attacker records the model's output — predictions, confidence scores, embeddings, or generated text. This input-output dataset becomes the training data.
Using the collected data, the attacker trains their own model to mimic the original. The clone learns to produce the same outputs for the same inputs — a functional copy.
Training a state-of-the-art model costs millions of dollars in compute, data collection, and engineering time. A model extraction attack can replicate much of that value for a few hundred dollars in API calls.
The model represents years of research and millions in investment. A stolen copy lets competitors skip all of that work.
Once an attacker has a local copy, they can craft adversarial attacks offline with white-box access, then deploy those attacks against the original.
Embed detectable patterns in model outputs that prove ownership if the model is stolen.
Add small amounts of noise to outputs that do not affect normal use but degrade extraction quality.
Detect unusual query patterns that indicate systematic probing of the model.
Restrict the number of queries per user to make large-scale extraction impractical.
If you can query a model enough times, you can steal it. A few hundred dollars of API calls can replicate millions of dollars of training investment. Any model exposed through an API is potentially vulnerable to extraction.
For hands-on practice, check out the AI Security course on GitHub.