AI Security Tools & Resources

LLM vulnerability scanner and red teaming framework from NVIDIA Research

Microsoft Python Risk Identification Tool for generative AI red teaming

LLM evaluation and red teaming framework with security testing plugins

Vulnerable LLM application for security training and testing

Interactive environment for testing prompt injection techniques

Microsoft tool for adversarial ML attack simulation

MITRE adversary emulation platform with ML attack plugins

Open-source LLM red teaming framework with guardrail testing capabilities

Agent-based automated red teaming at scale for AI systems

AI/ML vulnerability practice platform by Dreadnode

Fast vulnerability scanner with AI/LLM security templates

Static analysis tool with rules for AI/ML security vulnerabilities

Container and IaC scanner for AI model deployments

Security scanner for ML model files detecting malicious code

Automated vulnerability assessment for ML models

Dynamic application security testing specialized for AI systems

AI, Kubernetes, and API vulnerability testing platform

LLM-integrated extension for Burp Suite web security testing

Security toolkit for LLM applications with input/output validation

NVIDIA toolkit for building safe and controllable LLM applications

Self-hardening prompt injection detection service

LLM prompt injection scanner and detection system

Framework for adding validation and structured outputs to LLMs

WhyLabs toolkit for LLM monitoring and safety

Runtime protection layer for generative AI applications

IBM library for adversarial ML attacks and defenses

Framework for adversarial attacks on NLP models

Library for adversarial example generation and robustness testing

Python toolbox for adversarial attacks with multiple backends

Machine learning security library for attack and defense

Framework for multi-turn attacker-target simulations in LLM systems

GCG-based jailbreak attack generator for LLM security testing

Benchmark for evaluating automated red teaming and LLM safety

Comprehensive benchmark for LLM trustworthiness

Comprehensive trustworthiness evaluation for GPT models

UK AI Safety Institute evaluation framework

Framework for evaluating language models including safety

Standardized benchmark for evaluating LLM jailbreak defenses

Stanford safety benchmark suite for holistic LLM evaluation

Safety benchmark for computer-use AI agents

Framework for evaluating LLM behavior and capabilities

Automated fuzzing framework for jailbreak testing

Fuzzing framework for LLM-based applications

Benchmark for evaluating LLM robustness to adversarial prompts

Automated fuzzing framework using genetic algorithms for AI model testing

Microsoft data protection and PII detection for AI systems

Differential privacy library for PyTorch training

Privacy-preserving machine learning library

Enterprise PII detection and redaction API

Automated PII detection for databases and data pipelines

Singapore AI governance testing framework and toolkit

ML lifecycle platform with security and governance features

ML experiment tracking with security and collaboration features

Version control for ML projects with security features

Signing and verification for ML model supply chain

Supply chain security framework for ML artifacts

Software supply chain security framework

SBOM generation for ML containers and dependencies

Safe serialization format for ML models preventing code execution

Open source LLM observability and security monitoring

ML observability platform with security monitoring

Open source ML monitoring and testing platform

ML model performance and safety monitoring

Testing and monitoring for ML models and data

LLM-powered web honeypot for threat intelligence gathering