Nearly every robustness tool assumes you already know your model: its task, its input shape, its class list, and ideally its gradients. Real engagements rarely start that way. Someone exports a network to ONNX, hands you the file, and the metadata is empty. If your tooling can't move from there, it doesn't help.
Introspect, infer, attack
An arbitrary ONNX file still tells you a great deal if you read it carefully. The graph's tensor shapes, its operator histogram, and its declared inputs and outputs are enough to make a confident guess at the task. An input shaped like a batch of three-channel images feeding a convolutional stack that ends in a flat vector of logits is a classifier; a box-like output with thousands of anchors is a detector; a small feature vector in and a few classes out is tabular; integer token inputs are a sequence model. From the inferred task, you can synthesize a valid input and get a baseline prediction, all before running a single attack.
You cannot assume gradient access on a model you were handed. So the test battery has to work without it.
Why gradient-free attacks are the right default
This is where the black-box literature earns its keep. Score-based attacks like Square Attack (Andriushchenko et al., 2020) use random search over structured perturbations and require only the model's output scores, no gradients, no differentiability, and notably no vulnerability to gradient masking, the failure mode that has quietly invalidated many published white-box evaluations. Decision-based methods like HopSkipJump go further, needing only the final label. Because these attacks treat the model as an oracle, they run against an arbitrary ONNX session with no assumptions about its internals.
That property is what makes an autonomous battery possible. Mirage's ONNX autopilot introspects the file, classifies the task, synthesizes an input, and then runs a forward-pass-only sequence: a noise-robustness sweep across an increasing perturbation budget, a score-based random search for a minimal evasion, and, when the estimator is available, a genuine Square Attack. Every finding is tagged with the corresponding MITRE ATLAS technique, so a result is not just a number but a named adversary behavior a security team can reason about.
What it does and doesn't tell you
A gradient-free first pass is exactly that, a first pass. It establishes whether a model is trivially fragile and quantifies how fragile, within a query budget, without any cooperation from the model owner. It is the fast, honest triage step. Deeper white-box analysis follows when you have the access to do it. But the triage is what turns 'we have a model and no idea if it's robust' into a concrete, prioritized starting point.