• Stars
    star
    144
  • Rank 255,590 (Top 6 %)
  • Language
    Python
  • License
    MIT License
  • Created about 4 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

pretrained BERT model for cyber security text, learned CyberSecurity Knowledge

SecBERT

standard-readme compliant Donate

中文说明 | English

SecBERT is a BERT model trained on cyber security text, learned CyberSecurity Knowledge.

Table of Contents

Downloading Trained Models

SecBERT models now installable directly within Huggingface's framework:

from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("jackaduma/SecBERT")

model = AutoModelForMaskedLM.from_pretrained("jackaduma/SecBERT")


tokenizer = AutoTokenizer.from_pretrained("jackaduma/SecRoBERTa")

model = AutoModelForMaskedLM.from_pretrained("jackaduma/SecRoBERTa")


Pretrained-Weights

We release the the pytorch version of the trained models. The pytorch version is created using the Hugging Face library, and this repo shows how to use it.

Huggingface Modelhub

Using SecBERT in your own model

SecBERT models include all necessary files to be plugged in your own model and are in same format as BERT.

If you use PyTorch, refer to Hugging Face's repo where detailed instructions on using BERT models are provided.

Fill Mask

We proposed to build language model which work on cyber security text, as result, it can improve downstream tasks (NER, Text Classification, Semantic Understand, Q&A) in Cyber Security Domain.

First, as below shows Fill-Mask pipeline in Google Bert, AllenAI SciBert and our SecBERT .

cd lm
python eval_fillmask_lm.py

Downstream-tasks

TODO


Star-History

star-history


Donation

If this project help you reduce time to develop, you can give me a cup of coffee :)

AliPay(支付宝)

ali_pay

WechatPay(微信)

wechat_pay

License

MIT © Kun

More Repositories

1

awesome_LLMs_interview_notes

LLMs interview notes and answers:该仓库主要记录大模型(LLMs)算法工程师相关的面试题和参考答案
1,126
star
2

CycleGAN-VC2

Voice Conversion by CycleGAN (语音克隆/语音转换): CycleGAN-VC2
Python
521
star
3

Vicuna-LoRA-RLHF-PyTorch

A full pipeline to finetune Vicuna LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the Vicuna architecture. Basically ChatGPT but with Vicuna
Python
207
star
4

Recurrent-LLM

The open-source LLM implementation of paper: RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text. AI 写小说,AI写作
Python
152
star
5

CycleGAN-VC3

Voice Conversion by CycleGAN (语音克隆/语音转换):CycleGAN-VC3
Python
137
star
6

LAS_Mandarin_PyTorch

Listen, attend and spell Model and a Chinese Mandarin Pretrained model (中文-普通话 ASR模型)
Python
121
star
7

ChatGLM-LoRA-RLHF-PyTorch

A full pipeline to finetune ChatGLM LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the ChatGLM architecture. Basically ChatGPT but with ChatGLM
Python
121
star
8

NLP4CyberSecurity

NLP model and tech for cyber security tasks
Jupyter Notebook
75
star
9

ThreatReportExtractor

Extracting Attack Behavior from Threat Reports
Python
74
star
10

Alpaca-LoRA-RLHF-PyTorch

A full pipeline to finetune Alpaca LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the Alpaca architecture. Basically ChatGPT but with Alpaca
Python
54
star
11

nude-detect

Porn Content Pic or Video Recognization
Python
38
star
12

location_clustering

用户地理位置的聚类算法实现—基于DBSCAN和Kmeans的混合算法
Python
25
star
13

awesome_NLP-Interview-Notes

nlp_interview notes and answers: 该仓库主要记录 NLP 算法工程师相关的面试题和参考答案
18
star
14

AI-WAF

AI driven Web Application Firewall
Python
18
star
15

apk-view-tracer

Apk-view-tracer is a trigger tool for Android Dynamic Analysis and can be used in android anti-virus dynamic analysis.
Python
18
star
16

drowsiness-detection

打瞌睡检测,通过检测眼皮对眼球的遮挡程度,判定是否打瞌睡😂
Python
17
star
17

HomoglyphAttacksDetector

Detecting Homoglyph Attacks with CNN model using Computer Vision method
Jupyter Notebook
11
star
18

RepackagedAppDetector

Detect re-packaged app on Android based on fuzzy hash of instructions in dex
8
star
19

Loss-Function-In-PyTorch

Loss Function in PyTorch
Jupyter Notebook
7
star
20

WindowsStoreCrawler

crawl windows application from windows store on windows 8
C#
7
star
21

SpeakerRecognition-ResNet-GhostVLAD

Utterance-level Aggregation For Speaker Recognition In The Wild, using a "thin-ResNet" trunk architecture, and a dictionary-based NetVLAD or GhostVLAD layer to aggregate features across time, that can be trained end-to-end
Python
7
star
22

PrivacyLeakAdvancedDetection

Privacy Leak and Behavior Detect on Android based on method call graph
Java
6
star
23

audio_classification_models.pytorch

audio/voice classification in pytorch implementations
5
star
24

GANs-implementation

GAN models implementation repo
Python
4
star
25

jackaduma

personal profile
4
star
26

speaker_recognition_models.pytorch

speaker recognition / speaker verification models in pytorch implementation
4
star
27

DotNetAppGuard

Decompile &Static Analysis Dot Net App by using java
Java
4
star
28

Speech-Transformer-PyTorch

Python
4
star
29

jackaduma.github.io

CSS
4
star
30

django-cache-machine-mongoengine

Automatic caching and invalidation for Django & Mongodb. using models through the mongoengine ORM.
Python
4
star
31

py-recommender-framework

Recommender Framework implemented by python
Python
4
star
32

LangChain-OpenLLMs

Langchain-OpenLLMs with local knowledge library based on open source LLMs.
Jupyter Notebook
4
star
33

Annotated-Diffusion-Model

The Annotated Diffusion Model
Jupyter Notebook
3
star
34

SecCopilot

2
star
35

malicious-url-detection-with-ML

malicious url detection with machine learning
Python
1
star
36

awesome_AI_in_CyberSecurity_papers

awesome AI in CyberSecurity papers list
1
star
37

phishing-url-detection-with-ML

phishing url detection with machine learning
1
star
38

awesome_AI_in_Speech_papers

awesome AI in Speech papers
1
star
39

weak-password-detection-with-ML

weak password detection with machine learning
1
star