Skip to content
Fathom Layer
open_source

NVIDIA Garak

Specs not independently verified
Visit Site
NVIDIA GarakImage unavailable

Verdict

An open-source command-line vulnerability scanner for LLMs — it probes a model for jailbreaks, prompt injection, data leakage, toxicity and forced hallucination, then writes a structured report of what got through.

Where it wins, where it doesn't

Pros

  • Fully open-source (Apache 2.0), no vendor lock-in
  • Works against hosted APIs, Hugging Face models and local endpoints alike
  • Large, actively maintained probe library with machine-readable output

Cons

  • Command-line tool — results need security literacy to act on
  • Scanning hosted models costs API tokens and can hit rate limits
  • Identifies weaknesses only; remediation is on you
Ideal forTeams self-hosting an open-weight model before exposing it internallyRed-teamers and security engineers assessing an LLM deploymentCI pipelines that gate a model change on a security scan

Key Features

  • ✦CLI scanner combining dynamic and static probes
  • ✦Jailbreak, prompt-injection, toxicity, leakage and hallucination probes
  • ✦Adapters for OpenAI, Anthropic, Hugging Face, Ollama and more
  • ✦Machine-readable JSONL reports
  • ✦Extensible probe and detector plugins

Editorial note

Garak — "generative AI red-teaming and assessment kit" — is the closest thing the LLM world has to nmap or Metasploit, an analogy the project uses itself. You point it at a target (an OpenAI or Anthropic API, a Hugging Face model, a local Ollama or llama.cpp endpoint), it runs a battery of probes, and it produces a JSONL report marking which attacks scored a hit. It covers known jailbreak families (DAN and relatives), encoding and Unicode tricks that slip past naive filters, forced hallucination, toxicity and training-data leakage. It was created by Leon Derczynski and is now maintained by NVIDIA and the community under Apache 2.0. Two honest caveats: it finds problems, it does not fix them, and scanning a hosted model runs real inference — you pay for the tokens and can trip rate limits. Reading the output well takes some security literacy.

Frequently Asked Questions

Who is NVIDIA Garak for?↓
NVIDIA Garak is a fit for teams self-hosting an open-weight model before exposing it internally, Red-teamers and security engineers assessing an LLM deployment and CI pipelines that gate a model change on a security scan.
What are the drawbacks of NVIDIA Garak?↓
The trade-offs we record are: Command-line tool — results need security literacy to act on, Scanning hosted models costs API tokens and can hit rate limits and Identifies weaknesses only; remediation is on you.
What does NVIDIA Garak do well?↓
Fully open-source (Apache 2.0), no vendor lock-in, Works against hosted APIs, Hugging Face models and local endpoints alike and Large, actively maintained probe library with machine-readable output.

Alternatives to consider

See all alternatives →

Further reading

Featured badge

Building this product? Add the badge to your site to show it’s in the index.

<a href="https://fathomlayer.com/intelligence/cybersecurity/nvidia-garak" target="_blank" rel="noopener noreferrer"><img src="https://fathomlayer.com/fathom-badge.svg" alt="Featured on Fathom Layer" width="250" height="54" /></a>