The defensive side of adversarial ML has a long graveyard of methods that looked strong until someone evaluated them properly. The lesson from that graveyard is not that defense is hopeless, several approaches hold up, but that robustness is a measured property, not a claimed one, and every real defense trades something away to get it.
Adversarial training, and its price
The most reliable defense remains the most direct one: train on adversarial examples. Madry and colleagues framed this as solving the inner maximization of a min-max objective with PGD and minimizing over it, and models trained this way genuinely resist the attacks they were trained against. The cost is equally real, adversarial training is expensive, and it tends to reduce clean accuracy. TRADES made that tension explicit, giving a principled knob to trade natural accuracy against robustness rather than pretending the trade doesn't exist.
A defense that doesn't state what it costs, in accuracy, latency, or coverage, is a defense that hasn't been evaluated honestly.
Certified robustness
Empirical robustness says 'we couldn't break it with these attacks.' Certified robustness says 'no perturbation within this radius can change the prediction, provably.' Randomized smoothing (Cohen, Rosenfeld & Kolter, 2019) made certification practical at scale: wrap a base classifier in Gaussian noise, take a majority vote, and derive a certified L₂ radius from the vote margin. The guarantee is strong and the method is model-agnostic, but the radius is modest and inference costs multiply with the number of noise samples. It buys certainty at the price of compute.
Detection and preprocessing
Not every defense tries to make the model itself robust. Detection sits in front of it. Feature squeezing (Xu, Evans & Qi, 2018) compares a model's prediction on an input against its prediction on a squeezed version, reduced bit depth, spatial smoothing, and flags large disagreements as likely adversarial. Input preprocessing and statistical fingerprinting play a similar role. These are cheap and deployable as a proxy, which is why they anchor a runtime detection layer, but they are best treated as one line of defense in depth rather than a wall.
The part everyone skips: evaluation
The through-line of the last decade is that defenses fail in evaluation, not in principle. Athalye, Carlini, and Wagner showed that many published defenses worked only because they induced gradient masking, they broke the attacker's gradients rather than the attack, and fell immediately to attacks that don't rely on gradients. This is precisely why a serious evaluation includes gradient-free attacks like Square Attack and adaptive, parameter-free ensembles like AutoAttack.
- Adversarial training, strong empirical robustness; costs clean accuracy and compute.
- Randomized smoothing, provable guarantees; modest radius, multiplied inference cost.
- Detection & preprocessing, cheap, deployable; a layer, not a wall.
- Rigorous evaluation, the non-negotiable that keeps the other three honest.
Mirage's job is the last item, done well: attack the model the way a capable adversary would, including the gradient-free attacks that expose masking, and turn the result into the specific mix of training, certification, and detection that fits the deployment. Robustness isn't a product you buy once. It's a property you measure, engineer for, and keep measuring.