Software decision profile

NVIDIA Garak

An open-source command-line vulnerability scanner for LLMs — it probes a model for jailbreaks, prompt injection, data leakage, toxicity and forced hallucination, then writes a structured report of what got through.

CybersecurityNVIDIA GarakDecision profile Fathom Layer

Evidence statusSpecs not independently verifiedgithub.com

Visit site

Editorial assessment open source

Explore the trade-offs.

Where it stands outFully open-source (Apache 2.0), no vendor lock-inExplore strengths ↗Close details ↑

Works against hosted APIs, Hugging Face models and local endpoints alike
Large, actively maintained probe library with machine-readable output

What to weighCommand-line tool — results need security literacy to act onExplore limitations ↗Close details ↑

Scanning hosted models costs API tokens and can hit rate limits
Identifies weaknesses only; remediation is on you

Best suited toTeams self-hosting an open-weight model before exposing it internally · Red-teamers and security engineers assessing an LLM deployment · CI pipelines that gate a model change on a security scan

Read at the source

What NVIDIA Garak’s own pages state.

Each line was read on the vendor’s site and found word for word in the quoted page. It records what the vendor states, not a test by Fathom Layer.

License: Apache-2.0github.com Checked automatically · Oct 4, 2026

Where it wins, where it doesn't

Pros

Fully open-source (Apache 2.0), no vendor lock-in
Works against hosted APIs, Hugging Face models and local endpoints alike
Large, actively maintained probe library with machine-readable output

Cons

Command-line tool — results need security literacy to act on
Scanning hosted models costs API tokens and can hit rate limits
Identifies weaknesses only; remediation is on you

Ideal forTeams self-hosting an open-weight model before exposing it internallyRed-teamers and security engineers assessing an LLM deploymentCI pipelines that gate a model change on a security scan

Key Features

✦CLI scanner combining dynamic and static probes
✦Jailbreak, prompt-injection, toxicity, leakage and hallucination probes
✦Adapters for OpenAI, Anthropic, Hugging Face, Ollama and more
✦Machine-readable JSONL reports
✦Extensible probe and detector plugins

Editorial note

Garak — "generative AI red-teaming and assessment kit" — is the closest thing the LLM world has to nmap or Metasploit, an analogy the project uses itself. You point it at a target (an OpenAI or Anthropic API, a Hugging Face model, a local Ollama or llama.cpp endpoint), it runs a battery of probes, and it produces a JSONL report marking which attacks scored a hit. It covers known jailbreak families (DAN and relatives), encoding and Unicode tricks that slip past naive filters, forced hallucination, toxicity and training-data leakage. It was created by Leon Derczynski and is now maintained by NVIDIA and the community under Apache 2.0. Two honest caveats: it finds problems, it does not fix them, and scanning a hosted model runs real inference — you pay for the tokens and can trip rate limits. Reading the output well takes some security literacy.

Frequently Asked Questions

Who is NVIDIA Garak for?↓

NVIDIA Garak is a fit for teams self-hosting an open-weight model before exposing it internally, Red-teamers and security engineers assessing an LLM deployment and CI pipelines that gate a model change on a security scan.

What are the drawbacks of NVIDIA Garak?↓

The trade-offs we record are: Command-line tool — results need security literacy to act on, Scanning hosted models costs API tokens and can hit rate limits and Identifies weaknesses only; remediation is on you.

What does NVIDIA Garak do well?↓

Fully open-source (Apache 2.0), no vendor lock-in, Works against hosted APIs, Hugging Face models and local endpoints alike and Large, actively maintained probe library with machine-readable output.

Alternatives to consider

See all alternatives →

Continue the inquiry