• This repository has been archived on 13/Dec/2023
  • Stars
    star
    264
  • Rank 155,103 (Top 4 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created over 8 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Framework for Testing WAFs (FTW!)

Framework for Testing WAFs (FTW)

Build Status

Purpose

This project was created by researchers from ModSecurity and Fastly to help provide rigorous tests for WAF rules. It uses the OWASP Core Ruleset V3 as a baseline to test rules on a WAF. Each rule from the ruleset is loaded into a YAML file that issues HTTP requests that will trigger these rules.

Goals / Use cases include:

  • Find regressions in WAF deployments by using continuous integration and issuing repeatable attacks to a WAF
  • Provide a testing framework for new rules into ModSecurity, if a rule is submitted it MUST have corresponding positive & negative tests
  • Evaluate WAFs against a common, agreeable baseline ruleset (OWASP)
  • Test and verify custom rules for WAFs that are not part of the core rule set

Installation

  • git clone [email protected]:fastly/ftw.git
  • cd ftw
  • Make sure that pip is installed apt-get install python-pip
  • pip install -r requirements.txt

Running Tests with HTML contains and Status code checks only

  • Create YAML files that point to your webserver with a WAF in front of it
  • py.test test/test_default.py --ruledir test/yaml

Provisioning Apache+Modsecurity+OWASP CRS

If you require an environment for testing WAF rules, there has been one created with Apache, Modsecurity and version 3.0.0 of the OWASP core ruleset. This can be deployed by:

  • Checking out the repository: git clone https://github.com/fastly/waf_testbed.git
  • Typing vagrant up

Running Tests while overriding destination address in the yaml files to custom domain

  • start your test web server
  • py.test test/test_default.py --ruledir=test/yaml --destaddr=domain.com --port 443 --protocol https

Run integration test, local webserver, may have to use sudo

  • py.test test/integration/test_logcontains.py -s --ruledir=test/integration/

HOW TO INTEGRATE LOGS

  1. Create a *.py file with the necessary imports, an example is shown in test/integration/test_logcontains.py
  2. All functions with test* in the beginning will be ran by py.test, so make a function def test_somewaf
  3. Implement a class that inherits LogChecker
  4. Implement the get_logs() function. FTW will call this function after it runs the test, and it will set datetimes of self.start and self.end
  5. Use the information from the datetime variables to retrieve the files from your WAF, whether its a file or an API call
  6. Get the logs, store them in an array of strings and return it from get_logs()
  7. Make use of py.test fixtures. Use a function decorator @pytest.fixture, return your new LogChecker object. Whenever you use a function argument in your tests that matches the name of that @pytest.fixture, it will instantiate your object and make it easier to run tests. An example of this is in the python file from step 1.
  8. Write a testing configuration in the *.yaml format as seen in test/integration/LOGCONTAINSFIXTURE.yaml, the log_contains line requires a string that is a regex. FTW will compile the log_contains string from each stage in the YAML file into a regex. This regex will then be used alongside the lines of logs passed in from get_logs() to look for a match. The log_contains string, then, should be a unique rule-id as FTW is greedy and will pass on the first match. False positives are mitigated from the start/end time passed to the LogChecker object, but it is best to stay safe and use unique regexes.
  9. For each stage, the get_logs() function is called, so be sure to account for API calls if thats how you retrieve your logs.

Making HTTP requests programmatically

Although it is preferred to make requests using the YAML format, often automated tests require making many dynamic requests. In such a case it is recommended to make use of the py.test framework in order to produce test cases that can be run as part of the whole. Generally making an HTTP request is simple:

  1. create an instance of the HttpUA() class
  2. create an instance of the Input() class providing whatever parameters you don't want to be defaulted
  3. provide the instance of the input class to HttpUA.send_request()

For some examples see the http integration tests

More Repositories

1

pushpin

A proxy server for adding push to your API, used at the core of Fastly's Fanout service
Rust
3,654
star
2

js-compute-runtime

JavaScript SDK and runtime for building Fastly Compute applications
JavaScript
199
star
3

go-fastly

A Fastly API client for Go
Go
154
star
4

Viceroy

Viceroy provides local testing for developers working with Compute.
Rust
149
star
5

fastly-rails

Please visit https://github.com/fastly/fastly-ruby.
Ruby
143
star
6

cli

Build, deploy and configure Fastly services from your terminal
Go
139
star
7

fastly-magento2

Module for integrating Fastly CDN with Magento 2 installations
PHP
125
star
8

terraform-provider-fastly

Terraform Fastly provider
Go
119
star
9

Avalanche

Random, repeatable network fault injection
Python
104
star
10

fastly-exporter

A Prometheus exporter for the Fastly Real-time Analytics API
Go
98
star
11

fastly-ruby

A Fastly API client for Ruby
Ruby
91
star
12

compute-sdk-go

Go SDK for building Fastly Compute applications
Go
79
star
13

wafefficacy

Measures the effectiveness of your Web Application Firewall (WAF)
Go
76
star
14

fastly-py

A Fastly API client for Python
Python
76
star
15

sidekiq-prometheus

Public repository with Prometheus instrumentation for Sidekiq
Ruby
75
star
16

next-compute-js

Run Next.js on Fastly Compute
TypeScript
74
star
17

WordPress-Plugin

The Official Fastly WordPress Plugin
JavaScript
59
star
18

uslab

Lock-free slab allocator / freelist.
C
57
star
19

compute-starter-kit-rust-default

Default package template for Rust based Compute projects
Rust
50
star
20

go-utils

utils for go
Go
44
star
21

compute-actions

GitHub Actions for building on Fastly Compute.
JavaScript
41
star
22

insights.js

Real user monitoring of network timing signals using the Open Insights framework
TypeScript
40
star
23

compute-rust-auth

Authentication at Fastly's edge, using OAuth 2.0, OpenID Connect, and Fastly Compute.
Rust
38
star
24

waf_testbed

Chef Cookbook which provisions apache+mod_security+owasp-crs
HTML
35
star
25

fastlyctl

A CLI for managing Fastly configurations
Ruby
35
star
26

fastly2git

Create a git repository from Fastly service generated VCL
Ruby
32
star
27

token-functions

Example implementations for Fastly's token validation
Java
29
star
28

terrarium-rust-guest

The "http_guest" crate used by Fastly Labs Terrarium https://wasm.fastlylabs.com/
Rust
29
star
29

performance-observer-polyfill

๐Ÿ”Ž Polyfill for the PerformanceObserver API
TypeScript
29
star
30

compute-js-static-publish

Static Publisher for Fastly Compute JavaScript
TypeScript
27
star
31

waflyctl

Fastly WAF CLI
Go
27
star
32

fastly-magento

Magento Extension for working with the Fastly Content Delivery Network
PHP
26
star
33

terrarium-templates

Template and example projects for Fastly Labs Terrarium https://wasm.fastlylabs.com
C
26
star
34

libvmod-urlcode

urlencode/urldecode functions vmod
C
24
star
35

fastly-php

A Fastly API client for PHP
PHP
24
star
36

expressly

Express style router for Fastly Compute
TypeScript
24
star
37

compute-starter-kit-rust-static-content

Static content starter kit for Rust based Fastly Compute projects. Speed up your websites with a Compute application serving content from a static bucket, redirects, security and performance headers, and a 404 page.
Rust
24
star
38

remix-compute-js

Remix for Fastly Compute JavaScript
TypeScript
22
star
39

log4j_interpreter

A Rust library for evaluating log4j substitution queries in order to determine whether or not malicious queries may exist.
Rust
22
star
40

compute-starter-kit-javascript-default

Default package template for JavaScript based Fastly Compute projects
JavaScript
22
star
41

vcl-json-generate

A VCL module that allows you to generate JSON dynamically on the edge
VCL
21
star
42

compute-starter-kit-assemblyscript-default

Default package template for AssemblyScript based Fastly Compute projects
TypeScript
20
star
43

fanout-compute-js-demo

Fanout Fastly Compute JavaScript demo
TypeScript
17
star
44

cd-with-terraform

Practical exercises for InfoQ "Continuous deployment with Terraform" workshop
HCL
16
star
45

fastly-perl

A Fastly API client for Perl
Perl
16
star
46

fastly-rust

A Rust Fastly API client library.
Rust
16
star
47

mruby-optparse

Port of Ruby's OptionParser to mruby
Ruby
16
star
48

http-compute-js

Node.js-compatible request and response objects
TypeScript
16
star
49

compute-js-opentelemetry

An implementation of OpenTelemetry for Fastly Compute
TypeScript
15
star
50

compute-at-edge-abi

Interface definitions for the Compute@Edge platform in witx.
Rust
15
star
51

blockbuster

VCR cassette manager
Ruby
13
star
52

demo-fiddle-ci

Using Fastly Fiddle to enable CI testing of Fastly services
JavaScript
13
star
53

secretd

Secret storage server
Go
12
star
54

heroku-fastly

Heroku CLI plugin for Fastly
JavaScript
10
star
55

go-mtr

go wrapped mtr --raw
Go
10
star
56

security-use-cases

Placeholder for security related use cases and demos
HCL
10
star
57

fastly-js

A Fastly API client for JavaScript
JavaScript
10
star
58

terrctl

A command-line client for Fastly Terrarium. https://wasm.fastlylabs.com
Go
10
star
59

compute-starter-kit-javascript-openapi-validation

OpenAPI Validation Starter Kit for Fastly Compute (JavaScript)
JavaScript
10
star
60

vscode-fastly-vcl

A Visual Studio Code extension which adds syntax highlighting for Fastly Varnish Configuration Language (VCL) files.
TypeScript
10
star
61

http_connection_monitor

Monitors your outbound HTTP requests for number of requests made over a persistent connection.
Ruby
9
star
62

fastly-test-blog

Test application for learning Fastly's UI
Ruby
9
star
63

uap-vcl

uap-vcl is a VCL module which parses a User-Agent string
VCL
8
star
64

sigsci-splunk-app

Splunk app for Fastly (Signal Sciences)
Python
8
star
65

librip

Librip is a minimal-overhead API for instruction-level tracing in highly concurrent software.
Python
8
star
66

fastly_nsq

Public repository with a convenience adapter & testing classes for apps talking to NSQ
Ruby
8
star
67

altitude-nyc-abcd-workshop

Practical exercises for "ABCD: Always be continuously deploying" workshop at Altitude NYC 2017
HCL
7
star
68

dnstap-utils

dnstap utilities implemented in Rust
Rust
7
star
69

compute-starter-kit-typescript

A simple Fastly starter kit for Typescript
TypeScript
6
star
70

ember-anti-clickjacking

Anti-Clickjacking in Ember
JavaScript
6
star
71

compute-starter-kit-rust-beacon-termination

Beacon Termination package template for Rust based Fastly Compute projects.
Rust
6
star
72

Raikkonen

Rรคikkรถnen tests races.
C
6
star
73

compute-hibp-filter

Fastly Compute enrichment to detect compromised passwords
Go
6
star
74

js-compute-testing

Write JavaScript tests from Node.js, against a local or remote Fastly Compute application
TypeScript
6
star
75

diff-service

An experiment in powering Edge diff functionality from Google Cloud Functions
JavaScript
6
star
76

jlog-go

Go bindings for jlog
Go
6
star
77

compute-ll-hls

Fastly Compute application for LL-HLS playlist manipulation.
Rust
6
star
78

vmdebootstrap

wrapper around debootstrap to create virtual machine disk images
Python
6
star
79

compute-starter-kit-rust-empty

Empty package template for Rust based Fastly Compute projects
Rust
5
star
80

compute-starter-kit-go-default

Default package template for Go based Fastly Compute projects
Go
5
star
81

altitude-ci-cd-workshop

Practical exercises for "Building a continuous deployment pipeline" workshop at Altitude 2017
HCL
5
star
82

fastly-blocklist

Configure request blocking for a Fastly service.
Python
5
star
83

compute-starter-kit-javascript-queue

Queuing package template for JavaScript based Fastly Compute projects. Park your users in a virtual queue to reduce the demand on your origins during peak times.
JavaScript
5
star
84

serve-vercel-build-output

A runtime environment that executes output that targets the Vercel Build Output API on Fastly Compute
TypeScript
5
star
85

edgeml-recommender

POC: similarity search recommendation engine at the edge using only Fastly Compute & Rust
Rust
4
star
86

compute-starter-kit-rust-websockets

WebSockets starter kit for Fastly Compute (Rust)
Rust
4
star
87

altitude-LON-logging-workshop

Fiddle links & exercises for "Building an internal analytics platform with real-time logs" workshop at Altitude LON 2019
4
star
88

irc2slack

Python
4
star
89

Varnish-API

Perl extension for accessing varnish stats and logs
C
4
star
90

security-solutions-visualization-waf-bigquery-looker

4
star
91

compute-starter-kit-javascript-empty

Empty package template for JavaScript based Fastly Compute projects
JavaScript
4
star
92

waf-simulator-automation

Example of automating waf tests with Fastly's waf simulator
Go
4
star
93

url-shortener

A tool makes your message easier to read
Rust
4
star
94

fastly-lem

Automate the deployment of Live Event Monitoring
Go
3
star
95

compute-js-auth

OAuth 2.0 implementation for Fastly Compute, in JavaScript
JavaScript
3
star
96

wasm-workshop-altitude-ldn-2019

Workshop materials for the "WebAssembly outside the web" workshop
Rust
3
star
97

compute-rust-sentry

Send error reports from Rust Fastly Compute services to Sentry.
Rust
3
star
98

compute-starter-kit-javascript-expressly

A lightweight starter kit for Fastly Compute, demonstrating the expressly framework.
JavaScript
3
star
99

fastly-template-rust-nel

Package template for a Rust based Network Error Logging Fastly Compute service
Rust
3
star
100

next-compute-js-server

Implementation of Next.js Server class for Fastly Compute JavaScript
TypeScript
3
star