• Stars
    star
    120
  • Rank 295,983 (Top 6 %)
  • Language
    CSS
  • License
    GNU General Publi...
  • Created about 5 years ago
  • Updated over 3 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Anti-Automation System


Predator
Predator

Anti-Automation System

Introduction

Predator is a prototype web application designed to demonstrate anti-crawling, anti-automation & bot detection techniques. It can be used a honeypot, anti-crawling system or a false positive test bed for vulnerability scanners.

Warning: I strongly discourage the use of the demonstrated methods in a production server without knowing what they exactly do. Remember, only the techniques which seem usable according to the web application should be implemented. Predator is a collection of techniques, its code shouldn't be used as is.

The mind map below is a loosely made visualization of how the techniques demonstrated here can be implmented in a production environment.

workflow

Note: The numbers and factors in "Observation Phase" can be used to set a reputation to a client which then can be used a strong indicator of malicious activity once a threshold is hit.

Techniques Used

Bot Detection

User-Agent and Header Inspection

HTTP headers sent by bots are often in different order when compared to a real browser or lack altogether. Many bots disclose themselves in the User-Agent header for the sake of ethics while others don't send one at all.

Webdriver Detection

Most of the HTML mutation techniques described here can be bypassed with browser based frameworks such as selenium and puppeteer but they can be detected with various tests as implemented in isBot.js.

Resource Usage

Most of the bots only make requests to webpages and images but resources files such as .css are often ignored as they aren't downloaded by the HTTP implementation in use. Bots can be detected when the ratio of webpages/images and such resource files becomes higher than a predefined threshold.

Malformed HTML

A lot of HTML parsers used in crawlers can't handle broken HTML as browsers do. For example, clicking the following link in a browser leads to page_1 but affected parsers parse the latter value i.e. page_2

<a/href="page_1"/href="page_2">Click</a>

It can be used to keep off and ban crawlers without affecting user experience.

Invisible Links

Some links are hidden from users using CSS but automated progarms can still see them. These links can be used to detect bots and take a desired action such as banning the IP address.

Bait Links

When Predator suspects that the visitor is a bot, it generates random number of random links which direct to a page (limbo.php) containing more random links and this process keeps repeating.

Signature Reversing

Vulnerability scanners usually enter a payload and see if the webapp responds in a certain way. Predator can pretend to have a vulnerability by including exptected response i.e. signature within HTML.
Predator mimics the followiwng vulnerabilities at the moment:

  • SQL Injection
  • Cross Site Scripting (XSS)
  • Local File Inclusion (LFI)

This method makes it possible to set up a honeypot without actually hosting any vulnerable code and serves as a test bed for false positive testing.

Credits

PatheticGeek did all the front-end magic to make Predator look good.

More Repositories

1

roop

one-click face swap
Python
27,272
star
2

XSStrike

Most advanced XSS scanner.
Python
13,111
star
3

Photon

Incredibly fast crawler designed for OSINT.
Python
10,869
star
4

Arjun

HTTP parameter discovery suite.
Python
5,093
star
5

AwesomeXSS

Awesome XSS stuff
JavaScript
4,730
star
6

sd-webui-roop

roop extension for StableDiffusion web-ui
Python
3,349
star
7

Smap

a drop-in replacement for Nmap powered by shodan.io
Go
2,860
star
8

Striker

Striker is an offensive information and vulnerability scanner.
Python
2,213
star
9

be-a-hacker

roadmap for a self-taught hacker
1,839
star
10

ReconDog

Reconnaissance Swiss Army Knife
Python
1,776
star
11

Hash-Buster

Crack hashes in seconds.
Python
1,716
star
12

huepy

Print awesomely in terminals.
Python
1,462
star
13

Corsy

CORS Misconfiguration Scanner
Python
1,335
star
14

uro

declutters url lists for crawling/pentesting
Python
1,105
star
15

Silver

Mass scan IPs for vulnerable services
Python
1,024
star
16

Decodify

Detect and decode encoded strings, recursively.
Python
864
star
17

Diggy

Extract endpoints from apk files.
Shell
862
star
18

Blazy

Blazy is a modern login bruteforcer which also tests for CSRF, Clickjacking, Cloudflare and WAF .
Python
833
star
19

Breacher

An advanced multithreaded admin panel finder written in python.
Python
717
star
20

ote

Generate Email, Register for anything, Get the OTP/Link
Python
569
star
21

goop

Google Search Scraper
Python
564
star
22

Parth

Heuristic Vulnerable Parameter Scanner
Python
559
star
23

Zen

Find emails of Github users
Python
553
star
24

Bolt

CSRF Scanner
Python
532
star
25

Orbit

Blockchain Transactions Investigation Tool
Python
524
star
26

JShell

JShell - Get a JavaScript shell with XSS.
Python
506
star
27

MyPapers

Repository for hosting my research papers
Python
498
star
28

Cloak

Cloak can backdoor any python script with some tricks.
Python
479
star
29

nano

Nano is a family of PHP web shells which are code golfed for stealth.
PHP
431
star
30

sqlmate

A friend of SQLmap which will do what you always expected from SQLmap.
Python
424
star
31

SubGPT

Find subdomains with GPT, for free
Python
332
star
32

dump

Stuff that doesn't deserves its own repository.
Python
300
star
33

hardcodes

find hardcoded strings from source code
Python
273
star
34

Quark

Quark is a data visualization framework.
JavaScript
213
star
35

proxify

A python module for dumping usable proxies.
Python
152
star
36

Shiva

Improved DOS exploit for wordpress websites (CVE-2018-6389)
Python
112
star
37

velocity

DNS caching for humans
Python
89
star
38

ifnude

nudity detector that works
Python
83
star
39

zetanize

HTML form parser for humans.
Python
73
star
40

rewise

Google auto-complete wrapper
Python
73
star
41

Locky

Locky generates "really" strong yet easy to remember passwords.
Python
63
star
42

Entropy

Entropy is a (prototype) WAF driven by maths.
Python
54
star
43

meta

Explains and tests HTTP response headers
Python
39
star
44

fonetic-go

golang implementation of fonetic (https://github.com/s0md3v/fonetic)
Go
37
star
45

fonetic

assess pronounciblity of text
Python
33
star
46

shades

Shell
26
star
47

wl

convert case style of words
Go
21
star
48

s0md3v.github.io

my personal blog
SCSS
15
star
49

s0md3v

11
star
50

dishtance

JavaScript
6
star
51

.github

my default .github files
5
star
52

BG3-Mutant

Mod for Baldur's Gate 3 that adds a Mutant class to the game.
2
star