Awesome GPT + Security 
A curated list of awesome security tools, experimental case or other interesting things with LLM or GPT.
Contents
Attention
Here is A nice tool to Finetune ALL LLMs with ALL Adapeters on ALL Platforms!
Tools
Audit
- SourceGPT - prompt manager and source code analyzer built on top of ChatGPT as the oracle
- ChatGPTScanner - A white box code scan powered by ChatGPT
- chatgpt-code-analyzer - ChatGPT Code Analyzer for Visual Studio Code
- hacker-ai - An online tool using AI to detect vulnerabilities in source code
- audit_gpt - Fine-tuning GPT for Smart Contract Auditing
- vulchatgpt - Use IDA PRO HexRays decompiler with OpenAI(ChatGPT) to find possible vulnerabilities in binaries
Reconnaissance
- CensysGPT Beta - The tool enables users to quickly and easily gain insights into hosts on the internet, streamlining the process and allowing for more proactive threat hunting and exposure management
- GPT_Vuln-analyzer - Uses ChatGPT API, Python-Nmap, DNS Recon modules and uses the GPT3 model to create vulnerability reports based on Nmap scan data, and DNS scan information. It can also perform subdomain enumeration to a great extent
- SubGPT - SubGPT looks at subdomains you have already discovered for a domain and uses BingGPT to find more.
- Navi - A QA based Reconnaissance Tool with GPT
- ChatCVE - The ChatCVE Lang Chain App is an AI-powered devSecOps application
🔍 , for oganizations triaging and aggregating CVE (Common Vulnerabilities and Exposures) information. - ZoomeyeGPT - ZoomEyeGPT browser extension is a GPT-based Chrome browser extension designed to bring AI-assisted search experience to ZoomEye users.
- uncover-turbo - Realize a general-purpose natural language surveying and mapping engine, and open up the last mile from natural language to surveying and mapping grammar.
Offensive
- PentestGPT - A GPT-empowered penetration testing tool
- burpgpt - A Burp Suite extension that integrates OpenAI's GPT to perform an additional passive scan for discovering highly bespoke vulnerabilities, and enables running traffic-based analysis of any type.
- ReconAIzer - A Burp Suite extension to add OpenAI (GPT) on Burp and help you with your Bug Bounty recon to discover endpoints, params, URLs, subdomains and more!
- CodaMOSA - CodaMOSA is the paper code of CodaMOSA: Escaping Coverage Plateaus in Test Generation with Pre-trained Large Language Models. It implements a fuzzer combined with OpenAI API, aiming to alleviate the problem of stagnant coverage in traditional fuzz.
- PassGAN - A Deep Learning Approach for Password Guessing. HomeSecurityHeroes land a Product, and you can test how much time an AI would need to crack your password here.
- nuclei_gpt - Only need to submit the relevant Request and Response and the description of the vulnerability to generate a Nuclei PoC.
- hackGPT - Leverage OpenAI and ChatGPT to do hackerish things
Detecting
- k8sgpt - a tool for scanning your Kubernetes clusters, diagnosing, and triaging issues in simple English.
- cloudgpt - Vulnerability scanner for AWS customer managed policies using ChatGPT
- IATelligence - About IATelligence is a Python script that will extract the IAT of a PE file and request GPT to get more information about the API and the ATT&CK matrix related
Preventing
Social Engineering
- ChatGPT-Web-Setting-Funny-Abuse - Play with ChatGPT-Web and found the HTML rendering in description settings.
Reverse Engineering
- Gepetto - About IDA plugin which queries OpenAI's gpt-3.5-turbo language model to speed up reverse-engineering
- gpt-wpre - Whole-Program Reverse Engineering with GPT-3
- G-3PO - A Script that Solicits GPT-3 for Comments on Decompiled Code
Investigation
- beelzebub - Go-Based Low-Code Honeypot Framework with Enhanced Security, Leveraging GPT-3 for System Virtualization
Fix
- wolverine - Auto fix the bugs in your Python Script/Code
Assessment
- falco-gpt - AI-generated remediations for Falco audit events
- selefra - an open-source policy-as-code software that provides analytics for multi-cloud and SaaS.
- openai-cti-summarizer - openai-cti-summarizer is a tool for generating threat intelligence summary reports based on OpenAI's GPT-3.5 and GPT-4 API
Cases
Experimental
- Lost in ChatGPT's memories: escaping ChatGPT-3.5 memory issues to write CVE PoCs
- I built a Zero Day virus with undetectable exfiltration using only ChatGPT prompts
- Experimenting with GPT-3 for Detecting Security Vulnerabilities in Code
- We put GPT-4 in Semgrep to point out false positives & fix code
- A Practical, AI-Generated Phishing PoC With ChatGPT
- Capturing the Flag with GPT-4
- I Used GPT-3 to Find 213 Security Vulnerabilities in a Single Codebase
- Using ChatGPT to generate encoder and supporting WebShell
- Using OpenAI Chat to Generate Phishing Campaigns -- Include Phishing Platform
- Chat4GPT Experiments for Security
- GPT-3 use cases for Cybersecurity
Academic
- GPT-4 Technical Report -- OpenAI's own security assessment and mitigation of the model
- Ignore Previous Prompt: Attack Techniques For Language Models -- Pioneering work of Prompt Injection
- More than you've asked for: A Comprehensive Analysis of Novel Prompt Injection Threats to Application-Integrated Large Language Models
- RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
- Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
- Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
- Can We Generate Shellcodes via Natural Language? An Empirical Study
Blogs
- Dissecting redis CVE-2023-28425 with chatGPT as assistant
- Security Code Review With ChatGPT
- ChatGPT happy to write ransomware, just really bad at it
- Create ATT&CK Groups Knowledge Base
Fun
- shortest prompt that will enable GPT to protect the secret key
- a CTF-like game that teaches how to bypass LLM using language hacks
GPT Security
Bypass Security Policy
- Chat GPT "DAN" (and other "Jailbreaks")
- ChatGPT Prompts for Bug Bounty & Pentesting
- promptmap - automatically tests prompt injection attacks on ChatGPT instances
Bug Bounty
- Building A Virtual Machine inside ChatGPT -- deprecated but interesting
- LangChain vulnerable to code injection -- CVE-2023-29374
Crack
- gpt4free -- Just API's from some language model sites.
- EdgeGPT -- Reverse engineered API of Microsoft's Bing Chat AI
Plugin Security
- SecureGPT – Dynamically test the security of your ChatGPT Plugins APIs (Free DAST for ChatGPT Plugins).
Contributing
Your contributions are always welcome! Please take a look at the contribution guidelines first.
If you have any question about this opinionated list, do not hesitate to open an issue on GitHub.
Thanks again for your contribution and keeping this community vibrant.