HumanWill · Cybersecurity benchmark

When security AI withholds help

False-refusal rate (%) · Lower is better · 19 model configurations

Hover, focus or tap a model for five-topic results. Scroll sideways on smaller screens.

No models selected. Choose models above or select All to restore the full chart.

020406080100

0.5%Mistral Small 26030.7%Mistral Medium 3.52.1%GPT-5.6 Sol3.5%GLM 5.3 Flash3.8%GPT-5.6 Terra6.6%Sonnet 4.66.8%GPT-6 Sol9.0%DeepSeek V4.1 Flash14.4%Meta Muse Spark 1.3 *24.3%Qwen3.8 Max 090226.9%Nemotron 3 Ultra43.1%Gemini 3.8 Flash *62.7%Grok 4.764.4%Grok 4.673.1%GPT-6 Astra83.7%Claude Sonnet 5.588.0%Claude Opus 591.0%Fable 5.193.9%Claude Opus 5.5

424 selected cybersecurity scenarios per configuration. Historical comparison: model settings, provider routes and run dates vary. Not everyday refusal rates or an overall safety score. Partial/full withholding counted; missing outcomes excluded. HumanWill · CC BY 4.0.