What's Changed

New plugins

• BEAST Attack Implementation by @erickgalinkin in https://github.com/leondz/garak/pull/728
• add probe for "grandma" attack by @leondz in https://github.com/leondz/garak/pull/740
• generator: lorem ipsum by @leondz in https://github.com/leondz/garak/pull/776

New features

• Feature: configurable plugins by @jmartin-tech in https://github.com/leondz/garak/pull/711
• Add JSON Path support for RestGenerator by @leondz in https://github.com/leondz/garak/pull/651
• Make banner line test dynamic by @jmartin-tech in https://github.com/leondz/garak/pull/670
• Tutorial for contributing a generator by @leondz in https://github.com/leondz/garak/pull/688
• add 'complete' message to report at end of run by @leondz in https://github.com/leondz/garak/pull/735
• support chat modality in attempts by @leondz in https://github.com/leondz/garak/pull/644
• add grandma substance synth probe, update docs and var names by @leondz in https://github.com/leondz/garak/pull/763
• update NVCF to support versioning, custom payload keys by @leondz in https://github.com/leondz/garak/pull/772
• WINDOWS & OSX SUPPORT: test by default on linux, osx, win by @leondz in https://github.com/leondz/garak/pull/793
• enable nested setting of plugin configs by @leondz in https://github.com/leondz/garak/pull/773
• always log nvcf payload by @leondz in https://github.com/leondz/garak/pull/778
• update ecoji for windows support by @jmartin-tech in https://github.com/leondz/garak/pull/787
• remove extraneous default params for nims that expect conservative pa… by @leondz in https://github.com/leondz/garak/pull/749
• require probes to match input modality of the generator by @jmartin-tech in https://github.com/leondz/garak/pull/738
• Improve cloze data for leakreplay probe by @leondz in https://github.com/leondz/garak/pull/708
• add CONTRIBUTING.md, update contributing guide in docs by @leondz in https://github.com/leondz/garak/pull/706
• add DEFAULT_PARAMS to base types that are configurables by @leondz in https://github.com/leondz/garak/pull/771
• atkgen - add configurability, allow config of which adversarial generator to use by @leondz in https://github.com/leondz/garak/pull/777
• enhancement: enable lazy setting of nested dicts by @leondz in https://github.com/leondz/garak/pull/775
• convert mutation_generator to use load_plugin by @jmartin-tech in https://github.com/leondz/garak/pull/795
• Add support for gpt-4o, gpt-4o-mini by @erickgalinkin in https://github.com/leondz/garak/pull/798
• bump transformers ver to min reqd for llama 3.1 by @leondz in https://github.com/leondz/garak/pull/800
• add mitigation strings, add test for string-based mitigation skipping by @leondz in https://github.com/leondz/garak/pull/762
• speedup report generation by using class attribs instead of instantiating by @leondz in https://github.com/leondz/garak/pull/707
• cached plugin enum by @jmartin-tech in https://github.com/leondz/garak/pull/768

Tuning & fixes

• add some actionable opinions about linting by @leondz in https://github.com/leondz/garak/pull/668
• handle target model returning None in atkgen by @leondz in https://github.com/leondz/garak/pull/678
• bump & fix black version by @leondz in https://github.com/leondz/garak/pull/686
• Bump/black 24.4.2 -- and apply formatting by @leondz in https://github.com/leondz/garak/pull/687
• convert module-level constant names to uppercase by @leondz in https://github.com/leondz/garak/pull/685
• Guard cli run with invalid or incomplete config by @jmartin-tech in https://github.com/leondz/garak/pull/677
• Catch generic exception from transformers in huggingface by @jmartin-tech in https://github.com/leondz/garak/pull/690
• stablize openai parallel by @jmartin-tech in https://github.com/leondz/garak/pull/689
• rest generator response must be a list by @jmartin-tech in https://github.com/leondz/garak/pull/693
• Set generator _call_model() and _generate() type hints; amend functions by @leondz in https://github.com/leondz/garak/pull/694
• typo in racial slur prompt by @leondz in https://github.com/leondz/garak/pull/697
• Generator streamlining, docs by @leondz in https://github.com/leondz/garak/pull/682
• Split NVCF generator into completion chat by @leondz in https://github.com/leondz/garak/pull/696
• Update the quality taxonomy by @leondz in https://github.com/leondz/garak/pull/703
• generator & detector structure validation by @leondz in https://github.com/leondz/garak/pull/702
• doc & inheritance fixes to DanInTheWildMini by @leondz in https://github.com/leondz/garak/pull/712
• Remove support for openai api v0 by @leondz in https://github.com/leondz/garak/pull/718
• restore req_template_json_object lost in refactor by @jmartin-tech in https://github.com/leondz/garak/pull/729
• utilize GarakException in cli by @jmartin-tech in https://github.com/leondz/garak/pull/730
• ensure report is printed in utf-8 for Windows by @jmartin-tech in https://github.com/leondz/garak/pull/732
• Linting - 20240612 by @leondz in https://github.com/leondz/garak/pull/733
• add garak preprint paper by @leondz in https://github.com/leondz/garak/pull/739
• only set generations if still default by @jmartin-tech in https://github.com/leondz/garak/pull/741
• return value from batch_decode is already a list by @jmartin-tech in https://github.com/leondz/garak/pull/743
• Handle openai 400 errors more gracefully by @leondz in https://github.com/leondz/garak/pull/744
• Refactor huggingface config support by @jmartin-tech in https://github.com/leondz/garak/pull/742
• rename class level probe doc_uri by @jmartin-tech in https://github.com/leondz/garak/pull/759
• retry openaicompatible requests if invalid content received by @leondz in https://github.com/leondz/garak/pull/761
• add colour theme config by @leondz in https://github.com/leondz/garak/pull/769
• reduce zalgo-ness by @leondz in https://github.com/leondz/garak/pull/770
• move REST generator exception into garak.exceptions by @leondz in https://github.com/leondz/garak/pull/779
• defensive coding: allow python generators more places by @leondz in https://github.com/leondz/garak/pull/782
• stop encoding.payloads accumulating each load by @leondz in https://github.com/leondz/garak/pull/780
• correct sorting for set json in plugin_cache.json by @jmartin-tech in https://github.com/leondz/garak/pull/794
• corrections to config format for TAP internal generators by @jmartin-tech in https://github.com/leondz/garak/pull/796
• add missing comma, extra test case in probes.test.Test by @leondz in https://github.com/leondz/garak/pull/797
• Bugfix/leakreplay output setting by @leondz in https://github.com/leondz/garak/pull/790

Full Changelog: https://github.com/leondz/garak/compare/v0.9.0.13...v0.9.0.14