• Stars
    star
    299
  • Rank 134,906 (Top 3 %)
  • Language
    C++
  • License
    MIT License
  • Created almost 2 years ago
  • Updated 6 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A DTrace on Windows Reimplementation

STrace

Steve's Tracer. A DTrace on windows syscall hook reimplementation. Think of this like a patchguard compatible SSDT hook, but without hacks. This doesn't support SSSDT (win32k apis), as the DTrace system call APIs themselves do not support this additional table. Zw* kernel apis can be traced in addition to all usermode SSDT syscalls.

Please read to the end of this readme if you're developing new plugins for STrace!

Functionality

Dtrace on windows supports multiple probe types. These include syscall, fbt, etw, profile, and maybe more. This reimplementation only re-implements the syscall and etw probe types. All other probe types are considered completely out of scope for this project, and will never be supported. Please feel free to fork the project if you wish to add additional probe types. The scope was limited to just syscall and etw probes due to the complexity of the other probe types and the systems they touch.

This reimplementation completely discards the D scripting language (note: NOT DLang the more popular modern language). The complexity of a VM in the kernel like the original DTrace implementation is unsuitable for this project. Instead, this implementation directly exposes the relevant C callbacks required to plug into the DTrace windows kernel interfaces. To enable 'hot-loading' of scripts a DLL based plugin system was used as a replacement for the VM + scripting evironment of the original dtrace. This plugin system accepts a 'normal' usermode DLL without any security checks enabled, or otherwise external dependencies and manually maps it into the kernel adress space. The plugin dll has exports which are invoked when kernel syscall callbacks occur. Kernel APIs are resolved via the normal import table (IAT) of the DLL, plugin DLLs link to ntoskrnl.lib and then the driver will resolve these APIs at load time, allowing any system apis to be called within plugin DLLs like normal. Performance of this plugin system is excellent as native code, rather than a script interpreter or a JIT, is directly executing between syscall ENTRY and RETURN. This design improves performance compared to the dtrace implementation provided by Microsoft.

Setting hooks is very simple, you get a routine to register/unregister a hook by api name, with a pre and post syscall callback. Callbacks have arguments and return values accessible as read only values. Return values can be spoofed in return probes and arguments can be modified in the entry probes, but the original syscall cannot (without hacks) be 'cancelled' - this API acts as an observer. When events occur and your hook callback fires, whatever you like can be done. Callbacks are synchonous, meaning if you delay execution in the entry callback by say sitting in a while loop then the syscall call will be delayed by that amount of time. Execution of the system call occurs just after return from the entry callback, and just before entry of the return callback. This system is fully patchguard compatible, however DSE must be disable as Microsoft unfortunately considers this type of kernel extension part of the NT kernel and so validates that the root signer is windows. This does not work with tricks like enabling custom kernel signers, DSE really must be off during kernel boot.

ValidationFlags=IMGP_LATEST_MS_ROOT_REQUIRED | IMGP_WINDOWS_ROOT_REQUIRED | IMGP_MS_SIGNATURE_REQUIRED 
Scenario=ImgSigningScenarioWindows

The Rust driver discards the DLL based plugin system used by the C++ driver and attempts to use a web assembly interpreter to host wasm scripts in the windows kernel instead. This is to be more similar to the original dtrace implementation which uses a VM, but with a better language than DLang. This provides sandboxing and other nice benefits. The POC is complete and works, but unfortunately is not usable due to performance issues with interpretering WASM with WASMI. A NT kernel compatible JIT based wasm engine would be required for this alternative design to be workable. It's included as a fun novelty, and might be usable for something else with some work.

Project Organization

This project is largely split between it's C++ and Rust components. The C driver is feature complete and should be preferred. At a high level the project has the following components

  • C++ Driver w/ DLL based plugins
  • Rust Driver w/ wasm script based plugins
  • C++ DLL plugin to monitor file deletes
  • C++ DLL plugin to log all system calls and arguments
  • Rust Wasm script plugin to test POC of wasm in NT Kernel
  • Rust Wasm script tester to mock execution of wasm plugins in usermode
  • Rust log symbolicator to symbolicate C++ driver stack traces. Doesn't use DIA SDK.

Installation & Setup

Build the driver and cli, move the files to the same folder as the script, then run the powershell script in install folder as admin:

./install_as_admin.ps1

Reboot, then select the STrace boot entry, and press F8 (not enter!) on this screen:

f8

That should bring up this screen, select boot with DSE disabled:

DSE

After successful boot, the CLI can be used to load and unload plugin DLLs to begin tracing. Two example plugins are provided. Read the details below in detail if you run into issues. You may need to reboot again the first time, and potentially manually set the STrace service to autostart at boot using a tool such as process hacker.

Installation Details

The original DTrace installation is here: https://techcommunity.microsoft.com/t5/windows-kernel-internals/dtrace-on-windows-20h1-updates/ba-p/1127929. The original DTrace requires Secure Boot and Virtualized Based Security to be configured, this is not the case with STrace as it doesn't implement the probe types (FBT) that require those features.

This project does the same operations as the installer, but via a simple powershell script instead of an MSI. The process is simple. First copy an apiset dll to system32 so that the kernel extension is activated. Then install a driver entry so that the driver main is executed at system startup for usermode communications. ApiSet dlls are digitally signed, so the original microsoft apiset was used, as required. This file is only provided in binary form and contains no logic other than to point the extension import to the implementing driver: ext-ms-win-ntos-trace-l1-1-0 -> dtrace.sys. See https://www.geoffchappell.com/studies/windows/win32/apisetschema/index.htm (ApiSetSchemaExtensions) for details on this mechanism.

STrace is loaded very early during kernel initialization, enabling Test Signing in the boot configuration database (BCD) is not sufficient. Driver signature enforcement (DSE) must be disabled for STrace to successfully load, and this must be done every boot manually. To facilitate this the install script creates an easy boot menu entry for the user to select. There is no BCD flag to allow DSE to be permanently disabled across reboots, the kernel specifically forbids this.

Forgetting to disable DSE on boot will win you a trip to the Automatic Repair menu upon restarting. You can try again without harm, if boot continues to fail the STrace driver will need to be manually removed from the drivers folder. Be aware that boot failures can cause the STrace service's autorun flag to be disabled by windows*, you may need to set this back to autorun if a boot failure occurs at any time.

Plugins

To develop your own plugins it's best to use one of the existing plugins as a base project. The visual studio projects are set with many very specific settings to generate free standing binaries with no dependencies. The non-default settings are too many to list, so simply copy one of the projects, and modify the code to add your own logic instead (https://stackoverflow.com/questions/884255/visual-studio-copy-project). If you create a useful plugin, please submit a PR ! The more plugins that are made the more useful this system is to everyone!

More Repositories

1

commando-vm

Complete Mandiant Offensive VM (Commando VM), a fully customizable Windows-based pentesting virtual machine distribution. [email protected]
PowerShell
6,656
star
2

flare-vm

A collection of software installations scripts for Windows systems that allows you to easily setup and maintain a reverse engineering environment on a VM.
PowerShell
5,733
star
3

capa

The FLARE team's open-source tool to identify capabilities in executable files.
Python
3,911
star
4

flare-floss

FLARE Obfuscated String Solver - Automatically extract obfuscated strings from malware.
Python
3,036
star
5

red_team_tool_countermeasures

YARA
2,629
star
6

flare-ida

IDA Pro utilities from FLARE team
Python
2,031
star
7

flare-fakenet-ng

FakeNet-NG - Next Generation Dynamic Network Analysis Tool
Python
1,677
star
8

speakeasy

Windows kernel and user mode emulation.
Python
1,290
star
9

SharPersist

C#
1,213
star
10

ThreatPursuit-VM

Threat Pursuit Virtual Machine (VM): A fully customizable, open-sourced Windows-based distribution focused on threat intelligence analysis and hunting designed for intel and malware analysts as well as threat hunters to get up and running quickly.
PowerShell
1,184
star
11

gocrack

GoCrack is a management frontend for password cracking tools written in Go
Go
1,101
star
12

flare-emu

Python
735
star
13

SilkETW

C#
641
star
14

stringsifter

A machine learning tool that ranks strings based on their relevance for malware analysis.
Python
636
star
15

Mandiant-Azure-AD-Investigator

PowerShell
593
star
16

Azure_Workshop

HCL
572
star
17

sunburst_countermeasures

YARA
560
star
18

Ghidrathon

The FLARE team's open-source extension to add Python 3 scripting to Ghidra.
Java
556
star
19

ReelPhish

Python
493
star
20

capa-rules

Standard collection of rules for capa: the tool for enumerating the capabilities of programs
489
star
21

iocs

FireEye Publicly Shared Indicators of Compromise (IOCs)
458
star
22

DueDLLigence

C#
450
star
23

FIDL

A sane API for IDA Pro's decompiler. Useful for malware RE and vulnerability research
Python
431
star
24

flare-wmi

C++
412
star
25

GoReSym

Go symbol recovery tool
Go
379
star
26

rvmi

rVMI - A New Paradigm For Full System Analysis
C
352
star
27

PwnAuth

Python
347
star
28

idawasm

IDA Pro loader and processor modules for WebAssembly
Python
332
star
29

ADFSpoof

Python
318
star
30

SimplifyGraph

IDA Pro plugin to assist with complex graphs
C++
303
star
31

ShimCacheParser

Python
258
star
32

OfficePurge

C#
256
star
33

msi-search

C
215
star
34

ioc_writer

Python
195
star
35

macos-UnifiedLogs

Rust
192
star
36

GeoLogonalyzer

GeoLogonalyzer is a utility to analyze remote access logs for anomalies such as travel feasibility and data center sources.
Python
191
star
37

flare-kscldr

FLARE Kernel Shellcode Loader
C
175
star
38

Vulnerability-Disclosures

C++
166
star
39

flare-qdb

Command-line and Python debugger for instrumenting and modifying native software behavior on Windows and Linux.
Python
161
star
40

flare-dbg

flare-dbg is a project meant to aid malware reverse engineers in rapidly developing debugger scripts.
Python
149
star
41

thiri-notebook

The Threat Hunting In Rapid Iterations (THIRI) Jupyter notebook is designed as a research aide to let you rapidly prototype threat hunting rules.
Python
146
star
42

route-sixty-sink

Link sources to sinks in C# applications.
C#
137
star
43

heyserial

Programmatically create hunting rules for deserialization exploitation with multiple keywords, gadget chains, object types, encodings, and rule types
YARA
130
star
44

dncil

The FLARE team's open-source library to disassemble Common Intermediate Language (CIL) instructions.
Python
124
star
45

flashmingo

Automatic analysis of SWF files based on some heuristics. Extensible via plugins.
Python
118
star
46

VM-Packages

PowerShell
117
star
47

Reversing

111
star
48

ioc-scanner-CVE-2019-19781

Indicator of Compromise Scanner for CVE-2019-19781
Shell
91
star
49

flare-bytecode_graph

Python
82
star
50

gocrack-ui

The User Interface for GoCrack
Vue
81
star
51

Volatility-Plugins

Python
80
star
52

unicorn-libemu-shim

libemu shim layer and win32 environment for Unicorn Engine
C++
70
star
53

citrix-ioc-scanner-cve-2023-3519

Shell
61
star
54

AuditParser

AuditParser
Python
56
star
55

remote_lookup

Resolves DLL API entrypoints for a process w/ remote query capabilities.
Visual Basic
54
star
56

synfulknock

Lua
48
star
57

SSSDKCMExtractor

Python
46
star
58

jitm

JITM is an automated tool to bypass the JIT Hooking protection on a .NET sample.
C++
43
star
59

goauditparser

Go
39
star
60

tf_rl_tutorial

Tutorial: Statistical Relational Learning with Google TensorFlow
Jupyter Notebook
39
star
61

macOS-tools

Python
38
star
62

apooxml

Generate YARA rules for OOXML documents.
Python
38
star
63

gootloader

Collection of scripts used to deobfuscate GOOTLOADER malware samples.
Python
36
star
64

capa-testfiles

Data to test capa's code and rules.
Max
35
star
65

pycommands

PyCommand Scripts for Immunity Debugger
Python
35
star
66

vocab_scraper

Vocabulary Scraper script used in FLARE's analysis of Russian-language Carbanak source code
Python
35
star
67

ARDvark

ARDvark parses the Apple Remote Desktop (ARD) files to pull out application usage, user activity, and filesystem listings.
Python
34
star
68

rvmi-rekall

Rekall Forensics and Incident Response Framework with rVMI extensions
Python
32
star
69

gocat

Provides access to libhashcat
Go
29
star
70

ics_mem_collect

Python
26
star
71

rvmi-qemu

QEMU with rVMI extensions
C
26
star
72

IDA_Pro_VoiceAttack_profile

Python
25
star
73

pulsesecure_exploitation_countermeasures

YARA
24
star
74

win10_auto

Python
23
star
75

rvmi-kvm

Linux-KVM with rVMI extensions
C
23
star
76

pivy-report

Poison Ivy Appendix/Extras
17
star
77

siglib

Python
15
star
78

vbScript_deobfuscator

Help deobfuscate VBScript
VBA
13
star
79

flare-gsoc-2023

Supporting resources and documentation for FLARE @ Google Summer of Code 2023
13
star
80

DFUR-Splunk-App

The "DFUR" Splunk application and data that was presented at the 2020 SANS DFIR Summit.
13
star
81

rpdebug_qnx

Python
11
star
82

mandiant_managed_hunting

Azure Deployment Templates for Mandiant Managed Huning
9
star
83

flare-floss-testfiles

Resources for testing FLOSS by the FLARE team.
C
6
star