• Probes can now be selected by MISP tag, e.g. owasp:llm01
• garak now automatically creates an HTML report on completion
• HTML reports can be grouped by module but also by probe tag category, so you can see e.g. top-level scores for prompt injection, hallucination, and so on
• logs now go to a dedicated log dir by default, to keep things clean
• new buffs: encoding.Base64, encoding.CharCode
• new generator: NeMo guardrails
• new probe: AutoDAN
• RealToxicityProbes now only loads local lists, much faster
• update OpenAI models list
• fix attempt parameter stability
• better logging of config params
• atk is now atkgen

Contributions from @erickgalinkin , @drazvan . Enjoy & Happy holidays! 🎅🎄

What's Changed

• Attempt no longer uses mutable type defaults by @leondz in https://github.com/leondz/garak/pull/360
• Add NeMoGuardrails generator (WIP). by @drazvan in https://github.com/leondz/garak/pull/345
• add test for mutable defaults bug in attempt.Attempt by @leondz in https://github.com/leondz/garak/pull/362
• refresh openai model name list by @leondz in https://github.com/leondz/garak/pull/363
• speed up realtoxicityprompts loading by @leondz in https://github.com/leondz/garak/pull/364
• Feature/digest report 231212 by @leondz in https://github.com/leondz/garak/pull/365
• Autodan by @erickgalinkin in https://github.com/leondz/garak/pull/367
• Auto-reporting by @leondz in https://github.com/leondz/garak/pull/368
• add guardrails doc connection by @leondz in https://github.com/leondz/garak/pull/369
• Feature/digest plugin descrs by @leondz in https://github.com/leondz/garak/pull/370
• Add Base64 and CharCode buffs by @erickgalinkin in https://github.com/leondz/garak/pull/372
• tidy buffs, add test for buff config loading by @leondz in https://github.com/leondz/garak/pull/376
• Feature/tag selection by @leondz in https://github.com/leondz/garak/pull/383
• set default for probe_tags in core config; use this as default cli arg by @leondz in https://github.com/leondz/garak/pull/386
• hitlogs should use same paths as other reporting. add test for this by @leondz in https://github.com/leondz/garak/pull/387
• Feature/reporting categories by @leondz in https://github.com/leondz/garak/pull/389

New Contributors

• @drazvan made their first contribution in https://github.com/leondz/garak/pull/345

Full Changelog: https://github.com/leondz/garak/compare/v0.9.0.9...v0.9.0.10